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Preface 


This  purpose  of  this  study  was  to  find  out,  first  hand, 
from  AMIS  users  (from  buying  activities  at  Air  Force  Systems 
Command)  how  the  current  system  could  be  improved  to  meet 
their  computer  automation  and  information  needs.  A  more 
precise  understanding  of  the  users  will  help  the  AMIS 
Program  Office  of  Air  Force  Systems  Command,  invest  their 
resources  on  efforts  that  will  provide  the  most  utility  to 
the  field. 

A  special  thanks  goes  to  the  AMIS  Program  Office;  in 
particular  Lt  Col  Alan  Whittle,  the  director,  and  Mr 
Cuthbert  Cornette,  the  deputy  director.  Their  strong 
commitment  to  improve  AMIS  for  the  user  is  most 
significantly  manifested  by  their  insistance  on  involving 
the  AMIS  user  in  the  design  of  on-going  AMIS  inprovement 
pro j ect s . 

I  am  extremely  indebted  to  Lt  Col  Curt  Cook,  my  thesis 
advisor,  for  giving  me  the  opportunity  to  support  a  current 
Air  Force  need.  His  guidance  and  support  were  invaluable. 

Most  of  all  I  would  like  to  thank  God,  not  only  for 
those  mentioned  above,  but  also  for  my  wife  Rachael  and  my 
patient  children. 


Garry  R.Y.  Shafovaloff 
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Abstract 

The  Air  Force  Systems  Command  (AFSC)  Acquisition 
Management  Information  System  (AMIS)  is  continually  being 
improved  to  better  meet  the  needs  of  its  users.  The  primary 
object  of  this  research  was  to  support  the  AFSC  AMIS  program 
office  and  the  AFSC  Contract  Automation  Working  Group  in 
determining  the  buying  activity  users'  level  of  satisfaction 
and  unmet  needs.  The  survey  results  will  provide  AFSC 
additional  information  on  what  improvements  to  AMIS  will 
provide  the  most  utility  to  the  users. 

The  atrudy-  found  that  that  36%  of  AFSC  buying  activity 
users  are  satisfied  with  AMIS.  Forty-four  percent  (44%) 
indicated  dissatisfaction.  Users  believe  that  more  training 
is  needed,  that  the  system  should  be  designed  for  the 
infrequent  user,  and  that  input  redundancies  should  be 
reduced.  Contracting  officers  and  buyers  believe  that  AMIS 
needs  to  provide  more  utility  for  their  functions. 

Management  needs  analysis  tools. 

Many  of  the  users'  unmet  needs  will  be  provided  by  the 
Integrated  Distributed  System  (IDS) ,  a  major  AFSC  AMIS 
development  project.  All  improvements  to  AMIS  must  ‘'fit'1 
the  information  and  automation  needs  of  the  user  and  must 
contribute  to  an  improved  procurement  process. 

vi  i  i 


A  NEEDS  ASSESSMENT  OF  AIR  FORCE 


SYSTEMS  COMMAND  BUYING  ACTIVITY 
ACQUISITION  MANAGEMENT  INFORMATION 
SYSTEM  (AMIS)  USERS 


I .  Introduction 


Background 

DoD' s  procurement  automation  initiatives  should  be 
shaped  by  the  needs  of  its  acquisition  mission  and  by 
the  current  and  emerging  information  technologies. 
(5:1.3) 

The  Air  Force  Systems  Command  (AFSC) ,  Directorate  of 
Contracting  Automation  is  designing  major  improvements  to 
the  Acquisition  Management  Information  System  (AMIS) .  The 
information  technology  for  major  improvements  is  available. 
Daily  interface  with  the  field  and  a  formal  AMIS  user 
working  group  have  been  primary  sources  for  defining  AMIS 
users’  needs.  To  further  validate  that  current  initiatives 
will  support  user  needs,  the  Directorate  of  Contracting 
Automation  decided  to  survey  a  sample  of  buying  activity 
AMIS  users.  The  AFSC  Aeronautical  Systems  Division  (ASD) 
Directorate  of  Contracting  and  Manufacturing  administered  a 
survey  to  ASD  buying  personnel  January  through  February 
1989.  The  object  of  that  survey  was  to  determine  how 
satisfied  AMIS  users  were  with  the  Procurement  Management 
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System  (PMS) ;  a  subsystem  of  AMIS.  The  survey  results  were 
briefed  to  the  AFSC  Contracting  Directors  Conference  in 
April  1989  and  to  the  AFSC  Contract  Automation  Working  Group 
in  May  1989,  Both  groups  expressed  a  desire  to  have  a 
similar,  command-wide  survey  that  would  determine  the  needs 
of  AMIS  users.  The  interests  of  these  groups  coincided  with 
the  AFSC  Contracting  Automation  Directorate’s  (hereafter 
called  the  AMIS  program  office)  desire  to  further  validate 
that  current  AMIS  program  office  development  efforts  would 
meet  AMIS  users'  needs. 

Problem  Statement 

An  unacceptable  level  of  uncertainty  exists  regarding 
the  level  of  satisfaction  and  unmet  needs  of  current  AMIS 
users.  Current  AMIS  development  efforts,  the  most 
significant  being  the  Integrated  Distributed  System  (10:1), 
must  meet  user  needs.  In  addition,  user  acceptance  and 
satisfaction  with  changes  to  the  system  are  key  to  ensuring 
added  utility. 

Purpose  of  the  Study 

The  purpose  of  this  study  is  to  reduce  the  level  of 
uncertainty  regarding  user  satisfaction  and  unmet  needs 
through  surveying  a  sample  of  the  AMIS  users  within  buying 
activities  of  AFSC.  The  primary  management  question  to  be 
answered  for  the  AMIC  program  office  is: 
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Do  the  current  and  planned  AFSC  AMIS  development 
efforts  coincide  with  user-perceived  needs? 


Research  questions  relevant  to  answering  the  management 
question  and  that  can  be  investigated  by  a  survey  are: 

How  satisfied  are  AMIS  users  with  the  current  system? 
What  are  the  users'  unmet  needs? 


Scope 

This  study  does  not  attempt  to  determine  the 

satisfaction  and  needs  of  AMIS  users  at  AFSC  Air  Force  Plant 

Representative  Offices  (AFPROs)  or  any  other  users  except  at 

the  following  AFSC  buying  activities: 

Aeronautical  Systems  Division  (ASD) 

Wright-Patterson  AFB  Ohio 

Ballistic  Systems  Division  (BSD) 

Norton  AFB  California 

Electronic  Systems  Division  (ESD) 

Hanscom  AFB  Massachusetts 

Munitions  Systems  Division  (MSD) 

Eglin  AFB  Florida 

Space  Systems  Division  (SSD) 

Los  Angeles  AFB  California 

Eastern  Space  and  Missile  Center  (ESMC) 

Patrick  AFB  Florida 

Western  Space  and  Missile  Center  (WSMC) 

Vandenberg  AFB  California 

Rome  Air  Development  Center  (RADC) 

Griffiss  AFB  New  York 

Air  Force  Flight  Test  Center  (AFFTC) 

Edwards  AFB  California 
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Air  Force  Space  Technology  Center  (AFSTC) 

Kirtland  AFB  New  Mexico 

Not  included  in  this  study  were  the  Arnold  Engineering  and 
Development  Center  (AEDC) ,  the  Foreign  Technology  Division 
(FTD) ,  and  the  Air  Force  Office  of  Scientific  Research 
(AFOSR) .  These  organizations  are  exempt  from  using  AMIS  per 
AFSC  Regulation  70-13  (3:1) . 
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II.  Review  of  Related  Literature 

Because  the  objective  of  this  research  is  of  a 
practical,  rather  than  theoretical,  nature,  the  scope  of  the 
literature  review  was  limited  to  identifying  material 
directly  relevant  to  the  Acquisition  Management  Information 
System  and  the  questions  needed  for  the  survey. 

The  purpose  of  AMIS  as  specified  by  AFSC  Regulation 
70-13  is  to: 

a.  Respond  to  the  DOD  directed  Military  Standard 
Contract  Administration  Procedures  (MILSCAP) ,  DOD 
4105. 63-M,  (Dec  1977). 

b.  Support  the  HQ  AF  Contract  Management  Division 
contract  administration  and  payment  system. 

c.  Make  contract  information  available  to  all  AFSC 
levels  involved  in  the  acquisition  decision  making 
process  through  on-line  queries  and  hard  copy  reports. 

d.  Support  contract  writing  and  validate  contract 
information  as  it  is  entered  into  AMIS  through  an 
automation  device,  (3:1) 

To  integrate  the  user  into  the  continuous  process  of 
improving  AMIS,  AFSC  created,  in  1983,  the  Contract 
Automation  Working  Group  (CAWG) .  The  CAWG  initially  met 
quarterly  but  has  since  changed  to  meeting  every  other 
month.  Members  of  the  CAWG  include  representatives  from 
AMIS  buying  activities,  the  AFSC  Contract  Management 
Division,  the  Air  Force  Institute  of  Technology,  the 
Director  and  Deputy  Director  of  the  AMIS  program  office, 
information  resource  experts  from  AFSC,  and  the  Assistant  to 
the  Deputy  Chief  of  Staff  (DCS)  for  Contracting,  Air  Force 
Systems  Command.  The  CAWG  has  the  following  objectives: 
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a.  Solicit  user  needs. 

b.  Resolve  database  problems. 

c.  Prioritize  AMIS  modification  requests. 

d.  Review  written  AMIS  documentation. 

e.  Provide  channels  of  communication  to  facilitate 

exchange  of  AMIS  information  among  users. 

f.  Provide  feedback  to  the  AMIS  office  and  to  the  DOD 

MILSCAP  administrator  concerning  system  modification. 

g.  Assist  the  Director  of  Contracting  Automation. 

(1:26) 

Involving  the  user  in  the  design  process  "increases 
commitment  and  assures  accuracy  of  requirements 
specifications  (2:595)."  The  CAWG  field  members  represent 
the  users  to  the  AMIS  program  office.  They  are  not  only  the 
AMIS  program  office's  key  to  understanding  user  needs  but 
also  are  used  to  prioritize  many  development  efforts.  "The 
more  active  the  users  in  information  requirements 
determination  and  in  approval  of  user  interface  design,  the 
more  likely  they  are  to  accept  the  system  and  utilize  it 
appropriately  (2:595)  ."  The  field  CAWG  representatives  also 
act  as  change  agents.  "An  active  change  agent  can  assure 
better  communication  with  the  user  and  minimize  the 
possibility  of  misunderstanding  (2:594)."  They  are  depended 
on  to  accurately  communicate  AMIS  developments  to  the  field. 

At  a  cost  estimated  by  the  General  Accounting  Office 
(GAO)  to  be  $25.5  million,  the  Federal  Power  Commission 
(9:i)  developed  but  failed  to  successfully  implement  the 
Regulatory  Information  System.  The  purpose  of  the  system 
was  to  provide  computerized  access  to  current  energy  data  to 
agency  officials  across  the  nation.  The  GAO  found  several 
reasons  that  contributed  to  the  failure  of  the  Federal  Power 
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Commission  Regulatory  Information  System.  First,  user  needs 
were  not  clearly  defined.  This  was  complicated  by  lack  of 
communication  among  system  developers  and  intended  users. 
Second,  they  found  that  the  system  capabilities  were 
implemented  before  successful  demonstration.  Other  problems 
cited  include  lack  of  continuous  involvement  and  support 
from  top  management,  and  disruptions  in  both  organization 
and  personnel  (9:ii). 

A  past  study  prepared  for  the  Air  Force  Business 
Management  Center  in  1982  cited  the  following  problem  with 
AMIS: 

The  lack  of  interaction  between  AMIS  users  and  the  AMIS 
office  contributes  to  the  inability  of  the  AMIS  staff 
to  appreciate  user  needs  and  thereby  make 
clarifications  and  corrections  to  the  system  and  its 
procedures.  C6:4) 

The  Contract  Automation  Working  Group  and  this  survey 
are  positive  indications  that  the  AMIS  program  office  is 
committed  to  involving  users  in  the  design  of  continued 
improvements  to  AMIS.  This  survey  will  serve  as  an  initial 
baseline  from  which  to  measure  changes  in  user  satisfaction 
in  the  future. 
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III.  Methodology 


Research  Design 

This  descriptive  research  effort  attempts  to 
systematically  identify  facts  and  characteristics  (7:46)  of 
the  buying  activity  AMIS  users  that  are  relevant  to 
information  system  design  and  development.  The  respondents 
were  randomly  selected  from  each  buying  activity 
participating  in  the  survey.  Based  on  the  extensive  written 
comments  to  the  open-ended  questions  of  the  ASD  survey,  the 
interview  technique  was  determined  to  be  the  most 
appropriate  form  of  survey.  However,  the  mail  survey  was 
chosen  because  of  the  added  benefit  of  involving  the  most 
users  in  this  needs  validation  effort. 

The  survey  questions  were  developed  by: 

a.  Reviewing  "AFSC  Contracting  Automation  Update" 

newsletters  to  the  field, 

b.  Reviewing  Contract  Automation  Working  Group  minutes. 

c.  Reviewing  available  literature  on  AMIS. 

d.  Reviewing  MIS  development  literature. 

e.  Reviewing  proposed  questions  with  the  Director  and 

Deputy  Director  of  the  AMIS  program  office. 

f.  Observing  an  AFSC  CAWG  quarterly  meeting. 

g.  Reviewing  questions  with  CAWG  members  at  the 

Integrated  Distributed  System  Design  Conference. 
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h.  Reviewing  the  ASD  survey  responses  (8:133) . 

i.  Reviewing  a  GAO  study  of  a  large  government 
Management  Information  System. 

Many  of  the  questions  and  the  responses  were  developed  from 

responses  to  the  ASD  survey.  As  stated  by  Isaac  and  Michael 

in  Handbook  in  Research  and  Evaluation: 

One  of  the  best  ways  of  developing  good  objective 
questions  is  to  administer  an  open-ended  form  of  the 
question  to  a  small  sample  of  subjects  representative 
of  the  population  in  which  you  are  interested.  These 
more  lengthy  answers  provide  the  data  from  which 
objective-type  answers  are  derived.  (8:133-134) 

During  development,  the  questionnaire  reached  a  maximum 
of  18  pages  with  55  items.  The  CAWG  members  felt  that  many 
would  not  complete  the  questionnaire  because  it  would  take 
too  much  time,  so  with  recommended  deletions  and  question 
restructuring  the  questionnaire  was  reduced  to  nine  pages 
with  44  items.  According  to  Dillman  in  MAIL  AND  TELEPHONE 
SURVEYS:  The  Total  Design  Method.  questionnaires  with  more 
than  eleven  pages  and  125  items  can  be  expected  to 
experience  a  decrease  in  response  rate.  However,  there  is 
no  significant  difference  in  response  rate  for 
questionnaires  that  are  less  than  eleven  pages  in  length 
(4:55)  . 

To  increase  the  response  rate,  the  cover  letter 
(Appendix  A)  was  signed  by  Brigadier  General  Meyer,  Deputy 
Chief  of  Staff  for  Contracting,  Headquarters  Air  Force 
Systems  Command.  The  letter  was  one  third  of  one  page 
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(relatively  short),  and  content-wise,  to  the  point  (8:136). 
The  survey  was  mailed  to  the  field  on  28  and  31  July  1989. 

The  survey  (Appendix  A)  had  44  items,  one  of  which  (item 
#44)  was  an  open-ended  question.  The  items  were  grouped  in 
sections.  The  first  section  had  ten  questions  which 
specifically  addressed  the  Procurement  Management  System  of 
AMIS.  The  second  section  (items  #11-17)  addressed  AMIS  as  a 
whole,  to  include  PMS .  The  third  section  (items  #17-19) 
addressed  contract  automation  in  general.  Item  #20  asked 
the  respondent  to  rank  computer  applications  in  order  of 
importance  to  the  respondent.  Six  applications  were  listed 
with  two  "Other"  answers  intended  for  capturing  other 
potential  applications  that  were  not  listed.  The  purpose  of 
Item  #21  was  to  find  out  how  familiar  the  field  users  are 
with  the  "AFSC  Contracting  Automation  Update"  newsletter. 
Item  #22  was  asked  to  determine  how  aware  the  field  is  about 
Computer-Aided  Instruction  training.  The  purpose  of  items 
#23  and  #24  were  to  determine  how  familiar  the  field  is  with 
the  Automated  Contracts  Manager  position  and  the  Contract 
Automation  Working  Group,  respectively.  Items  #25  and  26 
were  asked  to  determine  how  familiar  the  field  users  are 
with  the  process  of  recommending  changes  to  AMIS, 
specifically  the  work  order  process.  Item  #27  asked  the 
respondent  to  prioritize  improvement  efforts  that  would  make 
AMIS  more  useful  to  the  respondent.  Nine  areas  of 
improvement  were  listed  with  one  "Other"  response  intended 
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for  capturing  any  improvement  efforts  that  were  not  listed. 
The  purpose  of  item  #28  was  to  determine  what  sources  are 
used  and  not  used  to  resolve  AMIS  problems.  Item  #29  was 
specifically  asked  to  determine  to  what  extent  the  field  has 
been  trained  via  the  established  training  processes. 

The  purpose  of  item  #30  was  to  determine  the  computer 
literacy  of  the  field  in  terms  of  familiarity.  Basic 
computer  terms,  more  advanced  computer  terms,  AMIS  specific 
terms,  and  contracting  automation  applications  were  listed. 
Items  #31-33  were  asked  for  the  purpose  of  determining  how 
much  of  a  user's  time  is  spent  with  AMIS  related  tasks. 

Item  #34  sought  to  determine  who  was  primarily  making  the 
inputs  to  PMS .  The  purpose  of  item  #35  was  to  determine 
where  the  closest  terminal  with  access  to  AMIS  is  with 
respect  to  the  respondent.  Item  #36  asked  the  respondent  to 
indicate  whether  or  not  the  location  of  the  terminal  was 
convenient  for  the  respondent.  Items  #37-44  were 
demographic  questions.  Item  #44  was  an  open-ended  question 
for  general  comments. 

Questionnaires  were  returned  by  mail  to  the  AMIS  program 
office  in  an  envelope  provided  with  the  survey.  Responses 
were  recorded  and  analyzed  with  the  STATISTIX  (7:9)  software 
package.  Frequency  and  cross-tabulations  were  performed  on 
the  data.  To  ensure  accuracy  of  the  data  entered,  a  scatter 
diagram  was  performed  on  each  variable  (e.g.  Question  1  by 
Question  1)  to  visually  identify  erroneously  entered  data. 
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When  an  outlier  was  observed,  the  data  was  reviewed  and 
corrected  as  required. 

Limitat ions 

One  of  the  limitations  was  that  although  the 
organization  charts  provided  by  each  contracting  activity 
were  the  most  currently  available,  some  of  the  selected 
survey  subjects  could  have  changed  jobs,  retired,  or 
separated  between  the  receipt  of  the  lists  during  mid-July 
1989  until  the  survey  was  released  on  28  July  1989,  Also, 
some  of  the  subjects  could  have  taken  leave  during  the 
survey  period.  These  factors  would  reduce  the  response 
rate.  Additionally,  the  response  period  was  during  the  next 
to  last  month  of  the  fiscal  year,  traditionally  a  very  busy 
time  for  contracting  activities. 

Another  limitation  of  this  survey  was  the  breadth  of 
coverage.  This  survey  was  administered  to  procurement 
clerks,  procurement  assistants,  contract  specialists  (called 
buyers/negotiators  in  the  survey  instrument) ,  contracting 
officers,  procurement  analysts,  and  procurement  managers. 

An  attempt  was  made  to  write  the  questions  so  that  they 
would  be  answerable  by  all  positions.  Only  one  survey 
recipient  called  regarding  the  applicability  of  some  of  the 
questions.  The  survey  subject  was  told  to  answer  as  many  of 
the  questions  as  possible. 

There  were  some  opportunities  for  bias  in  the 
development  of  the  survey  questions.  First,  as  the 
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facilitator  of  survey  development,  the  author  could  have 
inserted  bias  into  which  questions  were  selected  and  how  the 
questions  were  asked  due  to  his  procurement  experience  as  a 
contract  negotiator  at  Aeronautical  Systems  Division. 

Second,  the  author  had  to  remain  aware  of  the  biases  of  the 
AMIS  program  office  and  Contract  Automation  Working  Group. 
The  author's  observation  at  meetings  was  that  both  groups' 
desire  to  improve  AMIS  for  the  user  compensated  for  any 
personal  bias.  Another  general  limitation  has  to  do  with 
the  domain  from  which  the  questions  were  developed.  The 
AMIS  program  office,  the  Contract  Automation  Working  Group, 
the  "AFSC  Contracting  Automation  Update"  newsletter,  CAWG 
minutes,  and  the  author’s  personal  experience  as  a  contract 
negotiator  make  up  the  domain.  Not  included  in  the  domain 
were  direct  input  from  the  contracting  activity  Automated 
Contracts  Managers  who  are  not  also  members  of  the  CAWG.  It 
also  would  have  been  valuable  to  interview,  at  least  some, 
of  the  contracting  activity  directors  and  "users"  at  HQ 
AFSC. 
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IV.  Results  and  Discussion 


Overview 

The  survey  response  rate  after  18  days  was  50%.  The 
findings  presented  below  are  based  on  those  251  responses. 
After  the  cutoff  for  analysis  purposes,  25  additional 
completed  surveys  were  received,  increasing  the  actual 
response  rate  to  55%.  Findings  concerning  AFSC  AMIS  users 
as  a  group  are  statistically  significant  at  a  confidence 
level  of  90%  with  a  margin  of  error  of  plus  or  minus  5%. 
Because  of  ASD’ s  high  number  of  responses,  findings 
involving  ASD  are  statistically  significant  at  a  90% 
confidence  level  with  a  10%  margin  of  error.  Findings  with 
regard  to  buyers  (contract  specialists)  are  significant  at 
the  90%  confidence  level  with  a  margin  of  error  of  plus  or 
minus  8.5%.  No  other  contracting  activities  or  functions 
have  significance  at  the  90%  confidence  level.  However,  the 
author  did  note  that  there  were  few  significant  changes  in 
response  distributions  with  regard  to  skewness  or  peakedness 
after  100  responses  were  tallied.  This  indicates  that  the 
selection  of  subjects  was  effectively  random. 

Criteria  was  established  for  the  sake  of  consistently 
describing  frequencies  and  cross-tabulation  findings. 
Responses  for  Items  1  through  19  were  described  in  terms  of 
agree-disagree  or  satisf ied-not  satisfied  based  on  a  five 
point  Likert  scale  (8:142) .  For  all  items  differences  in 
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percentage  of  response  were  considered  markedly  different  if 
the  margin  of  difference  (excluding  neutral  responses)  was 
at  least  2:1  and  the  percentage  of  neutral  responses  was  35% 
or  less . 

Cross-tabulations  were  performed  with  buying  activity 
and  function  for  Items  1-19.  In  addition,  cross  tabulations 
with  acquisition  experience,  education,  and  user-perceived 
training  were  conducted  on  Item  11  (How  satisfied  with 
AMIS) ,  and  with  Items  1  and  2  which  had  a  relatively  higher 
correlation  with  Item  11  than  other  items. 

Survey  Results 

Item  1:  PMS  is  an  Effective  Tool  for  Tracking  the 
Overall  Status  of  Contract  Actions. 

As  depicted  by  Figure  1,  respondents  agreed  considerably 
more  than  disagreed  (by  a  margin  of  2:1)  that  the 
Procurement  Management  System  is  an  effective  tool  for 
tracking  the  overall  status  of  contract  actions. 
Approximately  59%  of  the  respondents  agreed,  12%  were 
neutral  and  29%  disagreed. 
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Cross  tabulations  for  Item  1  were  conducted  for  the 
variables  contracting  activity,  function,  length  of 
acquisition  experience,  education  level,  and  user-perceived 
extent  of  AMIS  training.  Reference  Figure  32  of  Appendix  B, 
all  contracting  activities  except  Ballistic  Systems 
Division,  Rome  Air  Development  Center  and  the  Air  Force 
Flight  Test  Center,  agreed  more  than  disagreed  that  PMS  is 
an  effective  tracking  tool.  Reference  Figure  33  of  Appendix 
B,  procurement  clerks,  procurement  assistants,  buyers, 
contracting  officers,  and  procurement  analysts  agreed  more 
than  disagreed  that  PMS  is  an  effective  tracking  tool. 
Contracting  officers,  as  a  group,  agreed  proportionately 
more  than  any  other  function.  M*..  .3ement  ,  which  includes 
group  leaders,  branch  chiefs,  division  chiefs  and  directors, 
excluding  the  director  of  the  contracting  activity, 
exhibited  approximately  the  same  levels  of  agreement  and 
disagreement  for  Item  1.  With  respect  to  acquisition 
experience  (in  years)  and  education,  respondents  as  a  group 
agreed  more  than  disagreed  that  PMS  is  a  useful  tracking 
tool.  For  both  variables,  reference  Figures  34  and  35  of 
Appendix  B,  the  level  of  agreement  was  approximately  60%, 
with  the  level  of  disagreement  increasing  and  the  level  of 
neutrality  decreasing  as  experience  and  education  levels 
increased.  Reference  Figure  36  of  Appendix  B,  respondents 
who  perceive  that  they  have  been  adequately  trained  (as 
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measured  by  Item  12  of  the  survey) ,  agreed  (68%)  that  PMS  is 
more  effective  for  tracking;  this  was  more  than  those  who 
indicated  that  they  were  neutral  (58%)  as  to  the  extent  of 
training,  or  those  who  indicated  that  they  were  not 
adequately  trained  (55%) . 

Item  2.  The  More  I  Learn  How  to  Use  PMS.  the  More 
Useful  it  has  Become  to  Me. 

The  respondents  to  the  survey  notably  agreed  more  than 
disagreed  (margin  of  2:1)  with  Item  2.  Fifty-two  percent 
(52%)  agreed,  23%  were  neutral,  and  25%  disagreed. 


Response  N 

Agree  49 

(  f 

Neutral  ob 
39 

Disagree  23 

TOTAL  244 

Figure  2.  Learning  More  About  PMS  Helps. 

Cross  tabulations  for  Item  2  were  conducted  for  the 
variables  contracting  activity,  function,  length  of 
acquisition  experience,  education  level,  and  user-perceived 
extent  of  AMIS  training.  Figure  37  of  Appendix  B,  indicates 
that  most  contracting  activities  agreed  that  the  usefulness 
of  PMS  increases  with  PMS  training.  Munitions  Systems 
Division  and  Space  Systems  Division  indicated  the  greatest 
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degree  of  agreement  (72%  and  69%  respectively) .  Electronic 
Systems  Division  indicated  the  greatest  degree  of 
disagreement  (48%) .  Figure  45  indicates  that  regardless  of 
function,  respondents  agreed  that  increased  PMS  training 
increases  the  usefulness  of  PMS.  As  a  group,  procurement 
analysts  agreed  the  most  (73%) ,  and  contracting  officers 
disagreed  the  most  (37%) .  Figure  46  of  Appendix  B  indicates 
that  regardless  of  length  of  acquisition  experience  (in 
years),  most  respondents  agree  with  Item  2.  Those 
respondents  with  less  than  three  years  of  experience  agreed 
the  most  (61%) ,  followed  closely  by  the  users  with  at  least 
10  years  experience  (56%) .  The  group  with  at  least  three 
years  experience,  but  less  than  ten  years,  had  the  lowest 
level  of  agreement  (43%)  and  along  with  those  with  ten  or 
more  years  of  experience,  had  the  highest  degree  of 
disagreement,  27%  and  30%  respectively.  With  regard  to 
user-perceived  extent  of  AMIS  training,  Figure  41  of 
Appendix  B  indicates  that  regardless  of  perceived  training, 
AMIS  users  agree  that  more  training  would  increase  the 
usefulness  of  AMIS  tools. 

Item  3;  PMS  is  More  Useful  to  Mv  Management  Than  it  is 
to  Me . 

As  indicated  by  Figure  3,  respondents  considerably 
agreed  more  than  disagreed  (margin  of  3:1)  that  PMS  is  more 
useful  to  their  management.  Fifty-six  percent  (56%)  agreed, 
27%  were  neutral  and  17%  disagreed. 
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Cross  tabulations  were  performed  for  the  demographic 
variables,  contracting  activity  and  function.  Reference 
Figure  42  of  Appendix  B,  all  contracting  activities  with  the 
exception  of  Missile  Systems  Division,  agreed  more  than 
disagreed  that  PMS  is  more  useful  to  management . 


Response  N 

Agree  by 

bo 

Neutral  b ( 
26 

Disagree  15 

mmmmmmk  e% 

TOTAL  245 

Figure  3.  PMS  More  Useful  to  Management. 

Munitions  Systems  Division  had  the  lowest  degree  of 
agreement  (28%)  and,  in  comparison  to  other  cross¬ 
tabulations  with  contracting  activity,  had  a  high  proportion 
of  neutral  responses  (48%) .  Analysis  of  function,  as 
depicted  in  Figure  43  of  Appendix  B,  indicates  that,  with 
the  exception  of  management,  all  functions  agreed  more  than 
disagreed  that  PMS  is  more  useful  to  management. 

Item  4:  A  One  Page  Summary  of  All  Contract  Actions  for 
Each  Buyer  Would  be  Very  Helpful  to  Me. 

Respondents  strongly  agreed  that  a  one  page  summary  of 
contract  actions  is  needed.  Seventy-one  percent  (71%) 
agreed,  18%  were  neutral,  and  11%  disagreed  as  indicated  in 
Figure  4 . 
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As  depicted  in  Figure  44  of  Appendix  B,  all  contracting 


activities  agreed  more  than  disagreed  that  a  one  page 
summary  of  contract  actions  is  needed. 


Response  N 

Agree  luu 

74 

Neutral  45 

17 

■■H  7% 

Disagree  9 

mm  4% 

TOTAL  245 

Figure  4.  One  Page  Summary  Needed. 

All  functions  indicated  a  high  level  of  agreement  at 
approximately  the  same  level  (73%) ;  contracting  officers 
exhibited  the  highest  level  of  agreement  (83%)  . 

Item  5:  More  Space  is  Needed  for  PMS  Contract  Status 
Comments  (e.cr.  to  explain  delays)  . 

Respondents  agreed  considerably  more  than  disagreed 
(margin  of  7:1)  that  more  space  is  needed  in  PMS  for 
explanatory  comments.  Sixty-nine  percent  (69%)  agreed,  21% 
were  neutral,  and  10%  disagreed  as  shown  in  Figure  5. 


Response  N 

Agree  lUo 

65 

Neutral  51 

14 

■■■■■  6% 

Disagree  11 

mmm  4% 

TOTAL  244 

Figure  5.  More  Space  Needed  for  Remarks. 
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Cross-tabulations  with  contracting  activity  and  function 
revealed  that  AMIS  buying  activity  users,  regardless  of 
where  they  work  or  their  function,  agreed  more  than 
disagreed  that  more  space  is  needed  for  explanatory  remarks. 
Reference  Figure  46  of  Appendix  B,  over  80%  of  the 
respondents  agreed  with  Item  5  at  Space  Systems  Division, 

Air  Force  Flight  Test  Center,  and  Air  Force  Space  Technology 
Center.  As  depicted  at  Figure  47  of  Appendix  B,  all 
functions  agreed  that  more  space  is  needed  for  explanatory 
comments . 

Item  6:  PMS  is  Used  by  Management  to  Evenly  Distribute 
Workload . 

Respondents  markedly  disagreed  more  than  agreed  (margin 
of  3:1)  that  management  uses  PMS  to  evenly  distribute 
workload.  As  indicated  at  Figure  6,  Fifty-five  percent 
(55%)  disagreed,  25%  were  neutral,  and  20%  agreed. 


Response  N 

Agree  11 

mmmm  4% 

39 

Neutral  ol 

b  o 

Disagree  l U 

TOTAL  244 

Figure  6,  Management  Assesses  Workload  with  PMS. 
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AFSC  buying  activities  disagreed  more  than  agreed  that 
managers  use  PMS  to  evenly  distribute  workload.  Ballistic 
Systems  Division  disagreed  the  most  at  69%;  Space  Systems 
Division  agreed  the  most  at  38%.  All  functions  disagreed 
more  than  agreed  with  Item  6  as  depicted  at  Figure  49  of 
Appendix  B.  Management  disagreed  more  than  agreed  by  a  very 
narrow  margin. 

Item  7;  Overall,  the  Standard  PMS  Lead-Times  for 
Milestones  are  Realistic. 

As  a  group,  respondents  disagreed  more  than  agreed  that 
PMS  milestone  lead-times  are  realistic.  However,  Figure  7 
indicates  that  the  proportion  of  disagreement  to  agreement 
responses  was  less  than  two  to  one;  51%  disagreed  and  27% 
agreed . 


Figure  7.  PMS  Lead-times  are  Realistic. 

With  respect  to  buying  activities,  as  shown  at  Figure  50 
of  Appendix  B,  ESD,  WSMC ,  RADC ,  AFFTC ,  and  AFSTC  strongly 
disagreed  that  PMS  milestone  lead-times  are  realistic.  ASD, 
MSD,  and  SSD  disagreed,  although  with  a  ratio  of  disagree  to 
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agree  responses  of  less  than  2:1.  BSD  and  ESMC  differed 
from  the  majority  of  buying  activities  by  agreeing  more  than 
disagreeing  (ratios  less  than  2:1)  that  PMS  milestones  are 
realistic.  Analysis  of  functions,  as  shown  at  Figure  51, 
indicates  that  procurement  analysts,  management,  and 
contracting  officers  disagreed  more  than  agreed  with  Item  7. 
Buyers  disagreed  more  than  agreed,  but  by  a  ratio  of  less 
than  2:1.  Procurement  clerks  and  procurement  assistants 
were  approximately  split  between  agreeing  and  disagreeing 
that  PMS  milestone  lead-times  are  realistic. 

Item  8:  Buyers  Should  Input  Their  Own  PMS  Data. 

As  reflected  by  Figure  8,  respondents  agreed  by  less 
than  a  2:1  margin,  that  buyers  should  input  their  own  PMS 
data.  Fifty-four  percent  (54%)  agreed,  14%  were  neutral, 
and  31%  disagreed. 


Response  N 

Agree  74 

C  Q 

Neutral  35 

35 

Disagree  41 

TOTAL  244 

Figure  8.  Buyers  Should  Input  PMS  Data. 

Figure  52  of  Appendix  B  indicates  wide  variability  among 
buying  activities  with  regard  to  whether  or  not  buyers 
should  input  their  own  PMS  data.  One  hundred  percent  (100%) 
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of  all  AFFTC  responses  agreed  with  Item  8.  BSD,  SSD,  and 
RADC  also  agreed  with  Item  8.  ESD,  MSD,  and  AFSTC  agreed, 
but  by  a  ratio  of  less  than  2:1.  WSMC  disagreed;  ASD  and 
ESMC  also  disagreed,  but  by  a  ratio  of  less  than  2:1.  As 
depicted  at  Figure  S3  of  Appendix  B,  contracting  officers, 
management,  procurement  analysts,  and  procurement  assistants 
agreed  that  buyers  should  input  their  own  PMS  data. 
Procurement  clerks  and  buyers  also  agreed,  but  by  a  ratio  of 
less  than  2:1. 

Item  9:  PMS  Provides  a  Complete  "Picture"  of  a  Buyer's 
Workload . 

Figure  9  shows  that  respondents  disagreed  considerably 
more  than  agreed  (margin  of  4:1)  that  PMS  provides  a 
complete  picture  of  a  buyer's  workload.  Seventy-three 
percent  (733s)  disagreed,  93s  were  neutral,  and  183s  agreed. 


Response  N 

Agree  11 

mmm  43s 

33 

mmmmmmmmmm 

Neutral  22 

mm mmmm  93s 

63 

U  j;  S  &  d  1  1  v 

TOTAL  244 

Figure  9.  PMS  Provides  a  Complete  Picture. 


Cross-tabulations  were  performed  with  contracting 
activity,  and  function.  All  contracting  activities  strongly 
disagreed  with  the  statement  that  PMS  provides  a  complete 


24 


picture  of  a  buyer’s  workload  (see  Figure  54) .  As  depicted 
in  Figure  55,  all  functions,  except  procurement  assistants, 
strongly  disagreed.  Management  disagreed  the  most  (89%) . 

Item  10:  Mv  Organization  Relies  Primarily  on  Another 


Figure  10.  PMS  is  Not  Primary  Tracking  System. 

With  the  exception  of  RADC ,  buying  activities  primarily 
use  PMS  for  tracking  contract  actions  (see  Figure  56) . 

Item  11:  Please  Indicate  Your  Degree  of  Satisfaction 
with  AMIS. 

Figure  11  indicates  that  respondents  as  a  group  can  not 
be  described  as  strongly  satisfied  nor  dissatisfied  with 
AMIS.  Although  strong  dissatisfaction  was  not  established, 
the  direction  of  opinion  can  be  said  to  at  least  favor 
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dissatisfaction.  Forty-four  percent  (44%)  indicated 
dissatisfaction,  20%  were  neutral,  and  36%  indicated 
satisfaction  . 


Response  N 

Satisfied  8 

mm  3% 

fv 

Neutral  4? 

OHt 

Dissatisfied.  41 

TOTAL  241 

Figure  11.  Satisfaction  with  AMIS. 

Cross-tabulations  were  performed  with  contracting 
activity,  function,  acquisition  experience,  education,  and 
perceived  training.  Reference  Figure  57  in  Appendix  B,  most 
contracting  activities  were  strongly  dissatisfied.  ESD  was 
the  most  dissatisfied  (68%) .  ASD  was  the  only  contracting 
activity  that  was  strongly  satisfied  (54%) .  SSD  and  ESMC 
were  approximately  even  in  responses  of  satisfaction  and 
dissatisfaction.  Reference  Figure  58  at  Appendix  B, 
Procurement  assistants  were  the  most  satisfied  function 
(73%) ;  the  most  dissatisfied  function  was  management  (57%) . 
As  depicted  in  Figure  59  at  Appendix  B,  the  proportion  of 
responses  between  satisfaction  and  dissatisfaction  were 
approximately  equal  for  those  with  less  than  three  years  or 
more  than  ten  years  acquisition  experience.  Those  with 
between  three  and  ten  years  experience,  were  moderately 
satisfied.  With  regard  to  perceived  training  (Figure  61  of 
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Appendix  B) ,  those  who  perceive  that  they  have  not  been 
adequately  trained  are  moderately  dissatisfied.  Those  who 
are  neutral  or  feel  they  have  adequate  training  showed 
moderate  satisfaction. 

Item  12:  I  have  been  Adequately  Trained  to  Use  Those 
AMIS  Tools  (e.cr.  PMS .  DP  Cl)  that  I  Need  to  Do  Mv  Job. 

Respondents  disagreed  more  than  agreed  that  they  have 
received  adequate  AMIS  training.  However,  the  ratio  of 
disagreement  to  agreement  was  less  than  2:1.  Reference 
Figure  12,  58%  disagreed,  11%  were  neutral,  and  31%  agreed. 


Response  N 


TOTAL  246 

Figure  12.  Users  Provided  Adequate  AMIS  Training. 

As  shown  by  Figure  62,  BSD  was  the  only  contracting 
activity  that  strongly  agreed  (62%)  that  adequate  training 
had  been  received.  [Note:  Even  with  this  strong  indication 
of  receiving  adequate  training,  BSD  still  was  strongly 
dissatisfied  with  AMIS  (55%).}  Most  other  buying  activities 
strongly  disagreed  that  adequate  training  had  been  received. 
Reference  Figure  63,  buyers,  contracting  officers,  and 
management  strongly  disagree  that  they  have  received 
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adequate  training.  Procurement  clerks,  assistants,  and 
analysts  agreed  that  they  had  received  adequate  training. 

Item  13:  I  Spend  a  Lot  of  Time  at  the  Terminal  Waiting 
for  a  Response  From  the  AMIS  Computer. 

Figure  13  indicates  that  respondents  agreed  considerably 
more  than  disagreed  with  Item  13  by  a  margin  of  2:1. 
Forty-five  percent  (45%)  agreed,  33%  were  neutral,  and  22% 
disagreed. 


Figure  13.  Wait  Too  Long  at  Terminal. 

With  regard  to  buying  activities,  only  BSD,  ESD,  and  SSD 
strongly  agreed  that  too  much  time  is  spent  waiting  at  the 
terminal  (see  Figure  64) .  AFFTC  indicated  an  extremely  high 
degree  of  neutrality  (72%) .  By  function  (see  Figure  65) , 
only  procurement  clerks  and  assistants  indicated  a  strong 
degree  of  agreement  with  Item  13. 

Item  14:  I  Receive  Sufficient  Advance  Notice  of  Changes 
to  AMIS. 

As  depicted  in  Figure  14,  respondents  disagreed 
considerably  more  than  agreed  (margin  of  almost  3:1)  that 
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they  receive  sufficient  advance  notice  of  changes  to  AMIS. 
Fifty-two  percent  (52%)  disagreed,  31%  were  neutral,  and  18% 
percent  agreed. 


Response  N 

Agree  4 
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39 

Neutral  ( ft 
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TOTAL  244 

Figure  14.  Advance  Notice  of  Changes  Received. 

Reference  Figure  66,  a  high  percentage  of  neutral 
responses  was  received  for  Item  14.  Except  for  MSD  and 
AFSTC ,  buying  activities  disagreed  that  they  receive 
sufficient  advance  notice  of  changes  to  AMIS.  AFFTC  and  SSD 
disagreed  the  most  (72%  and  69%  respectively) .  By  function 
(see  Figure  67) ,  there  was  moderate  to  strong  disagreement 
with  procurement  assistants  disagreeing  the  most. 

Item  15:  There  are  No  Problems  with  Most  Chancres  to 
AMIS. 

Respondents  disagreed  notably  more  than  agreed  (margin 
of  3:1)  that  there  are  no  problems  with  most  changes  to 
AMIS.  Forty-nine  percent  (49%)  disagreed,  35%  agreed,  and 
16%  agreed  as  indicated  in  Figure  15. 

Reference  Figure  68  at  Appendix  B,  among  the  contracting 
activities  there  was  general  disagreement  with  Item  15.  As 
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a  percentage,  RADC  disagreed  the  most  (83%) ;  MSD  and  AFSTC 
had  the  lowest  levels  of  disagreement  (28%  and  31% 
respectively) .  As  depicted  in  Figure  69  at  Appendix  B, 
there  was  generally  disagreement  among  the  functions. 
Procurement  analysts  were  approximately  even  in  opinion 
between  agreement  and  disagreement. 


Response  N 

Agree  4 

■  2% 

35 

Neutral  ob 
62 

Disagree  56 

TOTAL  243 

Figure  15.  No  Problems  with  Most  Changes  to  AMIS. 

Item  16:  AMIS  is  Easy  to  Use  (i.e,  "user  friendly") , 
Figure  16  indicates  that  respondents  disagreed 
considerably  more  than  agreed  (margin  of  almost  3:1)  that 
AMIS  is  user  friendly.  Fifty-two  percent  (52%)  disagreed, 
28%  were  neutral,  and  20%  agreed  with  Item  16. 
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As  depicted  at  Figure  70  of  Appendix  B,  only  BSD,  ESD, 
SSD  and  RADC  strongly  disagreed  that  AMIS  is  user  friendly. 
The  other  buying  activities  moderately  disagreed  or  were 
approximately  even  in  their  agreement  and  disagreement 
responses.  ESD  disagreed  the  most  (80%) .  Reference  Figure 
71,  procurement  analysts  strongly  agreed  that  AMIS  is  user 
friendly  (60%);  all  other  functions  disagreed,  with 
management  disagreeing  the  most  (67%)  . 

Item  17:  I  Believe  mv  Management  is  Committed  to 
Improving  the  Acquisition  Process  through  Improved  Use  of 
Computer  Applications  and  Tools  ■ 

By  a  margin  of  3:1  respondents  agreed  markedly  more  than 
disagreed  with  Item  17.  As  depicted  in  Figure  17,  63% 
agreed,  18%  were  neutral,  and  19%  disagreed. 


Response 

Agree 

Neutral 


N 
69 
87 
45 
28 

Disagree  20 
TOTAL  249 


28% 


35% 


18% 


11% 


Figure  17.  Management  Committed  to  Improvement. 


With  respect  to  buying  activity,  SSD  and  ESMC  had  the 
highest  level  of  agreement  (83%  and  81%  respectively)  with 
Item  17.  Noteworthy  is  that  zero  disagreement  responses 
were  received  from  SSD.  The  only  activity  that  did  not 
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exhibit  agreement  was  AFSTC .  AFSTC  responses  were 
approximately  split  between  agreement  and  disagreement. 
Procurement  analysts  showed  the  highest  level  of  agreement 
among  functions  (87%)  (see  Figure  73  of  Appendix  B) .  All 
functions  agreed,  with  procurement  clerks  agreeing  the  least 
(45%)  . 

Item  18:  The  Benefits  that  I  Realize  from  Computer 
Automation  (e.g.  PMS ■  DPCI)  Outweigh  Mv  Costs  (costs  in 
terms  of  time  and  effort) ■ 

Respondents  agreed  slightly  more  than  disagreed  that  the 
benefits  realized  by  computer  automation  outweigh  their 
costs.  However,  the  margin  of  difference  was  less  than  2:1. 
As  depicted  in  Figure  18,  42%  agreed,  22%  were  neutral,  and 
36%  disagreed. 


Response  N 
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Figure  18.  Benefits  of  Computer  Outweigh  Costs. 


Reference  Figure  74,  AFFTC ,  ESMC,  WSMC,  SSD,  and  ASD 
agreed  more  than  disagreed  that  the  benefits  of  using  the 
computer  outweigh  the  costs.  BSD  disagreed  the  most  (52%) . 
With  respect  to  function  (see  Figure  75) ,  only  buyers 
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disagreed  more  than  agreed  that  the  benefits  of  using 
computers  outweigh  the  costs. 

Item  19:  I  Believe  that.  Within  My  Position,  the  Tasks 
that  Should  be  Automated  have  been  Automated. 

As  shown  in  Figure  19,  the  respondents  agreed  more  than 
disagreed  that  the  tasks  that  should  be  automated  have  been 
automated.  However,  the  margin  of  agreement  to  disagreement 
was  less  than  2:1.  Forty-six  percent  (46%)  agreed,  20%  were 
neutral,  and  34%  disagreed  with  Item  19. 


Response  N 
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Figure  19.  Enough  has  been  Automated. 

With  respect  to  buying  activities,  ASD,  BSD,  SSD,  and 
ESMC  indicated  agreement  that  what  should  be  automated  has 
been  automated  (see  Figure  76) .  WSMC,  RADC ,  AFFTC ,  and 
AFSTC  exhibited  overall  disagreement  with  with  Item  19.  As 
shown  in  Figure  77,  procurement  clerks  and  assistants 
strongly  agreed  and  procurement  analysts  strongly  disagreed. 

Item  20:  Ranking  of  Procurement  Computer  Applications 
and  Tools  According  to  the  Importance  to  the  Respondent. 
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As  depicted  in  Figure  20,  Imaging  received  more  priority 


rankings  of  "1"  than  any  other  application.  Electronic 
submission  of  Commerce  Business  Daily  Notices  received  the 
second  most  rankings  of  "1",  followed  closely  by  an  on-line 
print  capability  and  DD  Form  1547  (Weighted  Guidelines) 
generation  capability.  The  on-line  document  locator  and 
on-line  debarred/ suspended  list  received  considerably  fewer 
"1"  rankings. 


Tool  N* 

a.  43 

b.  53 

c .  6 

mm  3  % 

d.  19 

■Mum  8% 

e.  45 

r .  59 

TOTAL  225 

*  N  =  Number 

of  "1"  rankings  received 

Rank 

#4  a.  Weighted  Guidelines  Generation 

#2  b.  Electronic  Submission  of  Commerce  Business 

1  Daily 

(CBD)  Notices 

#6  c.  On-line  Debarred/ Suspended  List 

#5  d.  On-Line  Contract  File  Locator 

#3  e.  On-line  print  capability 

#1  f.  Imaging  -  storing  contracts  on  computer 

Figure  20.  Ranking  of  Computer  Applications. 


Item  21:  Familiarity  with  the  Quarterly  "AFSC 
Contracting  Automation  Update"  Newsletter, 

As  shown  in  Figure  21,  a  significant  proportion  (733s)  of 
the  survey  respondents  are  not  familiar  with  the  newsletter. 
Nine  percent  <99s)  indicated  that  they  were  familiar  with  the 


newsletter,  but  had  not  seen  it  for  over  six  months;  17% 
indicated  that  they  were  familiar  and  had  at  least  skimmed 
the  letter  within  the  past  six  months. 


Response  N 

a.  182 

b.  23 

c  .  43 

TOTAL  248 

a.  Not  Familiar 

b.  Familiar,  but  have  not  seen  the  letter  in 
over  six  months 

c.  Familiar,  and  have  seen  the  letter  within 
the  past  six  months 


Figure  21.  Familiarity  with  AFSC  Newsletter. 


Item  22:  Awareness  of  Computer-Aided  Instruction  (CAI) 
Courses ■ 

A  significant  proportion  of  the  survey  respondents 
indicated  that  they  are  not  aware  of  CAI  courses.  As 
depicted  in  Figure  22,  74%  were  not  aware  and  26%  were 
aware . 
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Figure  22  . 

Awareness 

of  Computer-Aided 

Instruction . 

Item  23:  Familiarity  with  Automated  Contracts  Manager 


A  large  proportion  of  respondents  indicated  familiarity 


with  the  Automated  Contracts  Manager  (ACM)  position.  As 
shown  in  Figure  23,  7336  indicated  familiarity  and  27% 
indicated  that  they  were  not  familiar  with  the  ACM  position. 
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Figure  23. 

Familiarity  with  ACM. 

Item  24 

Familiarity  with  the  Contract 

Automation 

Working  Group  (CAWG) , 

As  shown  in  Figure  24,  respondents  indicated  that  they 
are  generally  not  familiar  with  the  CAWG. 
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c.  Familiar  with  CAWG  and  with  group's  purpose 


Figure  24.  Familiarity  with  CAWG. 


Fifty-five  percent  (55%)  are  not  familiar  at  all  with  the 
AFSC  Contract  Automation  Working  Group,  27%  had  heard  of  the 
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group  but  were  not  familiar  with  its  purpose,  and  18%  were 
familiar  with  the  group  as  well  as  its  purpose. 


Item  25:  Awareness  of  Work  Order  Process  to  Improve 
AMIS. 

As  shown  in  Figure  25,  87%  of  the  respondents  were  not 
aware  of  their  ability  to  initiate  work  orders  to  improve 
AMIS;  13%  were  aware. 
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Not  Aware 

TOTAL 
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Figure  25  . 

Familiarity 

with  Work  Order  Process. 

Item  26 

Submission 

of  Work  Orders . 

Ten  of  the  248  respondents,  4%,  indicated  that  they  had 
initiated  a  work  order.  Of  the  ten  who  indicated  that  they 
had  initiated  a  work  order  four  were  procurement  analysts, 
three  were  managers,  two  were  contracting  officers,  and  one 
was  a  buyer.  The  buyer  and  one  manager  wrote  in  the  margin 
next  to  Item  26  that  nothing  ever  happened  with  the  work 
order  they  initiated. 

Item  27:  Categorizing  Improvement  Efforts  in  Terms  of 
Level  of  Priority. 
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As  shown  in  Figure  26,  the  top  three  improvement  efforts 
that  should  be  a  high  level  priority  according  to  the 
respondents  as  a  group  were: 

1)  Providing  more  training 

2)  Making  the  input  process  easier 

3)  Improving  the  accuracy  of  data 


Item  28:  Sources  of  Help  with  Computer-Related  Tasks. 
As  depicted  in  Figure  27,  the  top  three  sources  of  help 
for  the  respondents  as  a  group  were: 


1)  Organization's  designated  automation  focal  point 

2)  Fellow  worker 

3)  AMIS  Manual  (AFSCM  70-390) 
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The  least  used  sources  are  the  local  and  AFSC  automation 


news 1 et  t  ers . 


Sources  N 
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Rank 

#3  a.  AMIS  Manual 
#7  b.  Local  automation  guide 
#5  c.  Material  from  training 
#2  d.  Fellow  worker  with  expertise 
#1  e.  Organization's  official  AMIS  focal  point 
#4  f.  Personal  "lessons  learned"  notes 
#6  g.  Phone  call  to  AMIS  Program  Office  (AFSC/PKQ) 
#9  h.  "AFSC  Contracting  Automation  Update" 
newsletter 

#8  i.  Local  automation  newsletter 


Figure  27.  Sources  of  Help. 


Item  29:  Types  of  AMIS  Training  Received. 

The  training  most  received,  as  indicated  by  Figure  28, 
is  Local  One-on-One  training.  Fifty-four  (54)  respondents 
wrote  the  word  "none",  even  though  it  was  not  provided  as  an 
alternate  answer.  Of  the  28  respondents  that  indicated 
"Other"  (excluding  "none"  responses) ,  13  wrote  that  they 
learned  on  their  own.  An  example  of  comments  within  this 
category  include  "through  a  fellow  worker,"  "trial  and 
error,"  "reading  the  AMIS  manual’"  or  a  "newsletter."  Nine 
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"in-house  training." 


"Other"  responses  wrote  references  to 
Twenty-five  (25)  respondents  did  not  answer  the  question. 
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TOTAL 
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a.  Local  one-on-one  training  as  identified  in  "AFSC 
Contract  Automation  Training  Guide" 

b.  CAI  -  Computer-Aided  Instruction  training 

c.  PMS/Query  training  at  the  AMIS  Program  Office 

d.  Contractor  furnished  training 

e.  •None  -  Fifty-four  respondents  wrote  "none" 

in  the  "Other"  space  or  margin 

f.  Other  (excluding  "none"  responses) 


Figure  28.  AMIS  Training  Received  by  Users. 


Item  30:  Familiarity  with  Computer-Related  Terms. 
Terms  were  classified  into  four  groups: 


Group  1 
Group  2 
Group  3 
Group  4 


Basic  computer  terms 

Moderately  complex  computer  terms 

AMIS  unique  terms 

Current  Development  and  Application  terms 


Respondents  had  a  high  degree  of  familiarity  with  Group 
1  terms  and  a  high  degree  of  unfamiliarity  with  Group  4 
terms.  With  regard  to  Group  2  respondents  as  a  whole  were 
less  familiar  than  familiar  but  by  a  margin  of  less  than 
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2:1.  Figure  29  identifies  the  relative  differences  in 
familiarity  between  and  within  groups. 
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Respondents’  answers  were  categorized  for  analysis 
purposes  as  follows: 

Category  1  -  Respondent  does  not  use  AMIS  at  all. 

Category  2  -  Respondent  uses  AMIS  more  than  0%  and 
less  than  or  equal  to  5%  of  work  week. 

Category  3  -  Respondent  uses  AMIS  more  than  5%  but 
less  than  or  equal  to  20%  of  week. 

Category  4  -  Respondent  uses  AMIS  more  than  20%  of  a 
work  week. 

As  depicted  in  Figure  78,  forty-four  percent  (44%)  of 
the  respondents  indicated  that  their  total  use  of  AMIS 
involves  more  than  20%  of  their  work  week.  Twenty-nine 
percent  (29%)  indicated  that  they  use  AMIS  more  than  5%  but 
less  than  or  equal  to  20%.  Figure  79  reflects  the 
combination  of  responses  by  function  for  Item  31  (data 
entry) ,  Item  32  (retrieving  data) ,  and  Item  33  (data 
analysis) .  A  significant  proportion  (87%)  of  the 
procurement  clerk  and  procurement  assistant  groups  interface 
with  AMIS  more  than  20%  of  their  work  week  (Category  4) . 
Approximately  the  same  proportion  of  buyers,  contracting 
officers,  and  management  within  their  respective  groups, 
interface  with  AMIS  either  more  than  5%  but  less  than  20% 
(Category  3) ,  or  more  than  20%  (Category  4) .  A  significant 
proportion  of  procurement  analysts  indicated  use  of  more 
than  0%  but  less  than  or  equal  to  5%  of  their  work  week. 

Item  34:  In  Your  Office.  Who  Primarily  Inputs  Data  into 

PMS? 
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As  depicted  in  Figure  30,  procurement  clerks  primarily 
input  PMS  data.  Fifty  percent  (50%)  indicated  that  clerks 
primarily  input  data,  17%  indicated  that  buyers  primarily 
input  PMS  data,  and  13%  indicated  that  procurement 
assistants  primarily  input  PMS  data.  Of  the  49  respondents 
that  answered  “Other",  21  (8%  of  all  respondents)  indicated 
that  both  procurement  clerks  and  procurement  assistants 
primarily  input  PMS  data. 

Response  N 

a.  122 

b.  43 

c .  32 

d.  49 

TOTAL  246 

a.  Procurement  Clerk 

b.  Buyer 

c.  Procurement  Assistant 

d.  Other  _ 

Figure  30.  Primarily  Inputs  PMS  Data. 

Item  35:  Where  is  the  Closest  Terminal  with  Access  to 
AMIS  Located  in  Relation  to  Your  Desk? 

As  shown  in  Figure  31,  respondents  indicated  that  36%  of 
the  terminals  with  access  to  AMIS  were  located  at  both  the 
procurement  clerks  workstation  and  at  the  respondents 
workstation.  The  88  responses  identified  in  Figure  31  do 
not  include  any  procurement  clerk  responses.  Nineteen 
percent  (19%)  indicated  that  the  terminals  were  located  at  a 
central  computer  workstation. 
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Figure  31.  Closest  Terminal  with  AMIS  Access. 


Item  36:  Is  the  Location  of  the  Closest  Terminal  with 
Access  to  AMIS  Convenient  for  You? 

Item  36  was  directly  referenced  to  Item  35  which  asked 
whether  or  not  the  location  of  the  closest  terminal  with 
access  to  AMIS  was  convenient  to  the  respondent . 
Respondents  notably  agreed  more  than  disagreed  that  the 
closest  terminal  with  access  to  AMIS  was  convenient  for 
them.  Seventy-two  percent  (72%)  agreed  and  24%  disagreed. 

Item  44:  Open-Ended  Question  Requesting  Comments  on 
AMIS. 

All  answers  were  evaluated  and  categorized.  The 
categories  receiving  the  most  written  comments  were: 

1.  More  training  needed. 

2.  Make  AMIS  more  user  friendly. 

3.  Improve  the  cumbersome  access  process. 

4.  Reduce  down-time  of  computer. 

5.  Design  system  for  infrequent  user. 

6.  Eliminate  input  redundancies. 


Appendix  C  contains  all  the  responses  to  Item  44. 
Sixty-four  percent  (64%)  of  the  managers  who  responded  to 
the  survey  provided  comments  in  Item  44.  Approximately  40% 
of  each  of  the  remaining  functions  provided  written 
comments.  Forty-two  percent  (42%)  of  those  who  indicated 
that  they  were  not  satisfied  with  AMIS  (Item  11)  and  36% 
that  indicated  that  they  were  satisfied,  provided  written 
comments.  Another  observation  was  that  47%  of  those  who 
provided  written  comments  also  felt  that  within  their  office 
that  tasks  that  should  be  automated  have  been  automated 
(Item  19) .  Only  36%  of  those  who  provided  written  comments 
felt  that  not  enough  has  been  automated  in  response  to  Item 
19. 

Discussion 

Of  all  AFSC  buying  activities,  only  users  at  ASD 
indicated  strong  satisfaction  with  AMIS.  Of  the  functions, 
procurement  assistants  are  satisfied  the  most  and  management 
is  satisfied  the  least.  Buyers,  contracting  officers,  and 
management  feel  that  they  are  not  adequately  trained. 

Users,  with  the  exception  of  procurement  assistants,  believe 
that  AMIS  is  not  user  friendly.  Changes  to  AMIS  are 
perceived  to  be  inadequately  tested  and  users  do  not  believe 
that  sufficient  advance  notice  of  changes  is  given.  Users 
believe  that  although  PMS  is  useful  for  tracking  contracting 
actions,  it  is  more  useful  to  "someone  in  upper  management." 
Most  contracting  activities  did  indicate  that  PMS  is  the 
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primary  tracking  tool  but  users  do  not  believe  that 
milestones  are  realistic.  Most  managers  do  not  use  PMS  to 
evaluate  workload  distribution  and  most  users  believe  that 
PMS  does  not  adequately  reflect  the  total  buyer  workload. 
Most  users  do  believe  that  if  they  had  more  PMS  training 
that  PMS  would  be  more  useful.  In  general,  buying  activity 
AMIS  users  believe  that  the  benefits  of  computer  automation 
outweigh  their  costs  (in  terms  of  time  and  effort) .  They 
also  believe  that  management  is  committed  to  improving  the 
acquisition  process  through  use  of  computer  applications  and 
tools . 

The  most  useful  tools  for  users  would  be  imaging 
(contract  in  computer) ,  electronic  submission  of  Commerce 
Business  Daily  notices,  and  on-line  print  capability.  The 
key  efforts  that  would  provide  the  most  benefit  to  users 
(for  Item  27)  would  be  providing  more  training,  making  the 
input  process  easier,  and  improving  data  accuracy.  The 
first  two  efforts  coincide  considerably  with  the  written 
comments  provided  in  Item  44. 

Most  users  are  not  familiar  with  the  "AFSC  Contracting 
Automation  Update"  newsletter,  the  AFSC  Contract  Automation 
Working  Group,  or  the  AMIS  work  order  process  (major 
elements  of  the  automation  improvement  process) .  Users  were 
familiar  with  their  Automated  Contracts  Manager  (their  key 
problem  solver) . 
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One-on-one  training  is  the  most  common  form  of  AMIS 
training  experienced  by  users.  Most  users  were  not  aware  of 
Computer-Aided  Instruction  training.  Their  most  common 
sources  of  help  with  computer  related  tasks  were  the 
Automated  Contracts  Manager,  a  fellow  worker,  and  the  AMIS 
manual . 

Most  users  are  very  familiar  with  basic  computer 
terminology.  They  are  familiar,  in  general,  with  more 
complex  computer  terms  and  AMIS  specific  terms.  They  are 
not  familiar  with  terms  concerning  current  developments 
(e.g.  Integrated  Distributed  System)  and  applications.  The 
most  frequent  users  are  procurement  clerks  and  procurement 
assistants.  Clerks  input  PMS  data  the  most  followed  by 
buyers.  Most  users  have  computer  terminals  (with  AMIS 
access)  at  their  workstation  or  conveniently  located  nearby. 
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Recommendations 


The  AMIS  program  office  can  best  serve  the  buying 
activity  AMIS  user  community  by  not  only  upgrading  the 
Acquisition  Management  Information  System  with  technological 
advances  (e.g.  Integrated  Distributed  System),  but  by  also 
committing  resources  to  solving  very  specific  user- 
identified  problems. 

Training  processes,  formal  and  informal,  should  be 
reviewed  to  determine,  not  only  how  more  training,  but  also 
more  effective  training  can  be  accomplished.  If  buying 
activity  AMIS  users  had  better  exposure  to  the  “AFSC 
Contracting  Automation  Update"  newsletter,  they  would  know 
that  solutions  either  exist  or  are  being  developed  to  solve 
many  of  their  problems.  Increasing  the  visibility  of 
current  efforts  to  solve  problems  would  lead  to  higher  user 
satisfaction  and  acceptance  of  AMIS.  Information  from  this 
letter  should  be  integrated,  if  not  already,  into  local 
buying  activity  formal  and  in-house  training  programs.  It 
is  important  that  this  letter  be  made  readily  available  as  a 
source  of  help  for  each  AMIS  user.  Coupled  with  buying 
activities  increasing  the  exposure  of  AMIS  users  to  this 
letter,  the  AMIS  program  office  should  either  initiate 
training  to  increase  the  computer  literacy  of  users  or  write 
the  newsletter  to  match  the  computer  literacy  of  the  average 
AMIS  user . 
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Another  recommendation  to  improve  training,  is  to 
develop  a  "quick  reference"  guide  for  AMIS  users.  Most 
commercial  software  packages  come  with  small  fold-out  cards 
that  briefly  explain  operative  terms,  commands,  and 
procedures.  Use  of  this  card  would  help  the  user  find  an 
answer  faster  in  contrast  to  searching  through  the  AMIS 
manual  or  trying  to  contact  the  local  Automated  Contracts 
Manager . 

The  Integrated  Distributed  System  (IDS')  is  being 
designed  to  eliminate  many  current  AMIS  user  problems.  This 
system  will  make  AMIS  easier  for  the  infrequent  user  through 
extensive  use  of  menus,  windowing,  and  help  screens  (10:1) . 
Members  of  the  Contract  Automation  Working  Group  at  the  July 
1989  Integrated  Distributed  System  Design  Workshop  reviewed 
a  prototype  module  of  the  system.  Many  of  their  suggestions 
were  used  to  improve  the  design,  specifically  in  terms  of 
user  friendliness.  As  part  of  the  IDS  implementation, 
databases  will  be  distributed  to  the  buying  activities.  As 
a  result,  AMIS  user  access  should  significantly  improve. 

Improved  training  and  an  effectively  designed  Integrated 
Distributed  System  will  significantly  increase  the  utility 
of  AMIS  for  the  user.  In  addition  the  AMIS  program  office 
should  commit  resources  to  resolve  some  key  unmet  user 
needs.  Primarily  for  procurement  clerks  and  assistants,  the 
DD  Form  350  input  process  should  be  made  easier  by 
reordering  input  items  in  a  sequence  logical  to  the  person 
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inputting  the  data.  The  input  process  should  be  revised  to 
eliminate  the  need  to  reinput  data  already  in  the  database. 
All  functions  would  benefit  from  having  a  one  page  summary 
of  a  buyer’s  contract  actions.  This  summary  would  reduce 
the  amount  of  unneeded  information  and  paper  that  buyers 
currently  receive  on  a  weekly  basis.  Decision  support  and 
analysis  tools  must  also  be  developed  for  buyers, 
contracting  officers,  and  management. 

All  of  the  above  changes  are  recommended  assuming  that 
they  will  contribute  to  improving  the  procurement  process 
and  not  just  the  "automation"  process.  Will  a  new 
application,  such  as  electronic  submission  of  the  Commerce 
Business  Daily  notice  reduce  procurement  lead-time?  Are  we 
improving  user  satisfaction  in  areas  that  will  result  in 
improved  processing  and  management  of  information? 

The  final  and  most  important  recommendation  is  that  Air 
Force  Systems  Command  assess  information  requirements  at  all 
levels  of  the  procurement  process.  Who  is  using  what 
information  for  what  decisions?  Current  information  system 
development  efforts  must  "fit"  the  information  needs  of  the 
users  and  improve  the  procurement  process . 


50 


Appendix  A:  AFSC  Contracting  Automation  Surve 


:?  pk 

Contracting 


DEPARTMENT  OF  THE  AIR  FORCE 

HEADQUARTERS  AIR  FORCE  SYSTEMS  COMMAND 
ANDREWS  AIF  FORCE  BASE  DC  20334-5000 


26  July  1989 


Automation  Survey 


'c  All  Survey  Participants 

1.  Me  need  your  help!  Me  are  in  the  process  of  designing  an  improved 
Acquisition  Management  Information  System  (AMIS)  to  better  meet  your  needs. 

2.  The  attached  survey  gives  you  the  opportunity  to  express  your  ideas 
about  how  we  can  best  make  AMIS  work  for  you.  Please  take  a  few  minutes 
to  complete  the  survey.  Your  answers  are  anonymous  and  will  be  kept 
completely  confidential. 

3.  Mark  your  answers  on  the  attached  questionnaire  and  return  It  in 
the  enclosed  envelope.  Should  you  have  any  questions,  call  Capt  Garry 
Shafovaloff,  Air  Force  Institute  of  Technology,  Wrlght-Patterson  AFB  OH, 
AUTOVON  -M37. 


4.  I  look  forward  to  your  response! 


l 


KCTjrnrrv 

Brigadier  Gener 
D(£/ Contracting 


1  Atch 

Questionnaire 
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The  Procurement  Management  System  (PMS)  Is  an  AMIS  capability  that  automates 
the  storage  and  processing  of  data  on  contract  actions  from  the  time  a  buyer 
begins  working  on  a  requirement  until  the  resulting  contractual  action  Is 
retired  and  the  contract  file  Is  destroyed 


QUESTIONS  1-10  SPECIFICALLY  ADORESS  PMS. 

(Circle  the  number  to  the  Immediate  left  of  your  answer.) 


1.  PMS  Is  an  effective  tool  for  tracking  the  overall  status  of  contract 
actions. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  OISAGREE 

2.  The  more  I  learn  how  to  use  PMS,  the  more  useful  It  has  become  to  me. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

3.  PMS  Is  more  useful  to  my  management  than  it  Is  to  me. 

STRONGLY  MILOLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

4.  A  one  page  summary  of  all  contract  actions  for  each  buyer  would  be  very 
helpful  to  me. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  OISAGREE 

5.  More  space  Is  needed  for  PMS  contract  status  comments  (e.g.  to  explain 
delays). 

STRONGLY  MILOLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

6.  PMS  Is  used  by  management  to  evenly  distribute  workload. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

7.  Overall,  the  standard  PMS  lead-times  for  milestones  are  realistic. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  OISAGREE 


8.  Buyers  should  Input  their  own  PMS  data. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 


52 


9.  PMS  provides  a  complete  “picture"  of  a  buyer's  workload. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  OISAGREE  5  OISAGREE 

10.  My  organization  relies  primarily  on  another  computerized  system,  other 
than  PMS,  for  tracking  contract  actions. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  OISAGREE 


QUESTIONS  11  -  17  ADDRESS  AMIS  AS  A  WHOLE,  to  include  PMS,  Distributed  Writing 
for  Contractual  Input  (DPCI),  and  all  other  AMIS  applications  that  you  may  use. 


11.  Please  indicate  your  degree  of  satisfaction  with  AMIS. 

VERY  VERY 

1  SATISFIED  2  SATISFIED  3  NEUTRAL  4  DISSATISFIED  5  DISSATISFIED 

12.  I  have  been  adequately  trained  to  use  those  AMIS  tools  (e.g.  PMS,  DPCI, 
queries)  that  I  need  to  do  my  job. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

13.  I  spend  a  lot  of  time  at  the  terminal  waiting  for  a  response  from  the 
AMIS  computer. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  OISAGREE  5  OISAGREE 

14.  I  receive  sufficient  advance  notice  of  changes  to  AMIS. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

J  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

15.  There  are  no  problems  with  most  changes  to  AMIS. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

16.  AMIS  is  easy  to  use  (e.g.  "user  friendly"). 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 
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QUESTIONS  17-19  ADDRESS  CONTRACTING  AUTOMATION  IN  GENERAL. 

17.  I  believe  my  management  Is  committed  to  Improving  the  acquisition  process 
through  improved  use  of  computer  applications  and  tools. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

18.  The  oenefits  that  I  realize  from  computer  automation  (e.g.  PMS,  OPCI) 
outweigh  my  costs  (costs  in  terms  of  time  and  effort). 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

19.  I  Believe  that,  within  my  position,  the  tasks  that  should  be  automated 
have  Been  automated. 

STRONGLY  MILDLY  MILDLY  STRONGLY 

1  AGREE  2  AGREE  3  NEUTRAL  4  DISAGREE  5  DISAGREE 

20.  The  following  are  examples  of  computer  applications  and  tools  that  could 
be  included  in  an  improved  contracting  automation  system. 

Please  rank  these  features  in  order  of  importance  to  you. 

(1-most  important,  2-second  most  Important  ...  etc.) 

Don’t  hesitate  to  add  candidate  computer  applications  and  tools  to  the 
list! 

a.  _  DO  1547  generation 

B.  _  Commerce  Business  Oaily  Express 

(Electronic  submission  of  CBD  notices) 

c.  _  On-line  Debarred/Suspended  List 

d.  _  On-line  Document  Locator  (find  stored  contract  files) 

e.  _  On-line  print  capability 

f.  _  Storing  contract  files  on  a  computer  that  would  provide  me 

ready  viewing  access  on  my  office  computer  (imaging). 

g.  _  Other  _ 

h.  _  Other  _ 

21.  The  quarterly  "AFSC  Contracting  Automation  Update"  newsletter,  which  has 
been  published  since  Nov  87,  provides  information  about  AFSC  contracting 
automation  (including  AMIS  and  BCAS).  Which  statement  below  best 
describes  your  knowledge  of  this  "Update?" 

a.  I  am  not  familiar  with  this  letter. 

b.  I  am  familiar  with  this  letter;  however,  I  have  not  seen  it  for  over 
six  months. 

c.  I  am  familiar  with  this  letter  and  have  at  least  skimmed  it  within 
the  last  six  months. 


54 


22.  Computer-Aided  Instruction  ( CAI )  courses  are  available  through  AMIS  and 
you  can  take  them  right  from  your  office.  Certificates  are  awarded  upon 
completion  for  each  of  the  following  courses: 

-An  Overview  of  AMIS 
-How  To  Use  AMIS 
-Procurement  Management  System 
-Contract  Close-Out  Procedures 
-Undefinitized  Document  Control 
-Contractual  Document  Guidance 
-Data  Base  Queries/S2Ku  Natural  Language 
-Prices  (Price  History  Data  Base  System) 


Here  you  aware  of  CAI  training? 

a.  YES 

b.  NO 


23.  AFSCR  70-13  requires  that  each  contracting  activity  appoint  an  Automated 
Contracts  Manager  (ACM)  to  act  as  the  automation  focal  point  for  AMIS. 
Are  you  familiar  with  this  position  within  your  organization? 

a.  YES 

b.  NO 


24.  Are  you  familiar  with  the  AFSC  Contract  Automation  Working  Group? 
a.  I  am  not  familiar  at  all  with  the  group. 

o.  I  have  heard  of  the  group  out  am  not  familiar  with  what  it  does, 

c.  I  am  familiar  with  the  group  and  with  its  purpose. 


25.  Are  you  aware  that  you  can  initiate  work  orders  to  improve  AMIS? 

a.  YES 

b.  NO 


26.  Have  you  ever  submitted,  a  work  order  to  improve  AMIS? 

a.  YES 

b.  NO 
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27.  Prioritize  the  following  efforts  with  respect  to  how  much  each  would 
improve  the  usefulness  of  AMIS  to  you. 

Use  the  following  scale: 

1  -  This  should  oe  a  high  level  priority. 

2  -  This  should  be  a  medium  level  priority 

3  -  This  should  be  a  low  level  priority 

4  -  Should  not  be  a  priority  at  all 

a.  _  Providing  more  training  In  AMIS  applications  and  tools 

b.  _  Making  the  input  process  easier 

c.  _  Improving  the  ability  to  acquire  reports  tailored  to  my  needs. 

d.  _  Improving  the  response  time  for  receiving  reports  (specifically 

change  from  overnight  to  real-time) 

e.  _  Ability  to  analyze  data  to  include  graphic  outputs  (e.g.  average 

time  between  milestones  viewed  on  bar  chart  graph) 

f.  _  Improving  the  accuracy  of  data  in  the  computer. 

g.  _  Increase  the  ability  to  enter  narrative  comments  (e.g.  milestone 

delay  code  remarks) 

h.  _  Improve  the  clarity  of  error  messages 

1.  _  Improve  User  Manuals 

j.  _ Other  _ 

28.  Which  of  the  following  sources  do  you  consult  when  you  need  help  with  your 
contracting  automation  related  tasks? 

(Circle  all  that  apply) 

a.  AFSCM  70-390  (AMIS  Manual) 

b.  A  local  contracting  automation  guide 

c.  Training  material 

d.  A  fellow  worker  who  is  not  an  officially  designated  contracting 
automation  focal  point. 

e.  A  person  within  my  organization  who  has  been  officially  designated  as 
a  focal  point  for  contracting  automation. 

f.  My  own  personal  "lessons  learned"  notes 

g.  Call  the  AMIS  Program  Office  (AFSC/PKQ) 

h.  “AFSC  Contracting  Automation  Update"  newsletter 

i.  Local  Automation  Newsletter 

j .  Other  _ 

29.  The  "AFSC  Contracting  Automation  Training  Guide"  describes  the  following 
types  of  training.  Please  indicate  which  training  you  have  had. 

a.  Local' one-on-one 

b.  Computer-Aided  Instruction 

c.  PMS/Query  training  at  AFSC/PKQ 

d.  Contractor  furnished  training 

e.  Other 
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30.  Please  indicate  how  familiar  you  are  with  the  terms  listed  below. 


Use  the  following  scale: 

1.  Very  familiar 

2.  More  than  familiar 

3.  Familiar 

4.  Slightly  familiar 

5.  1  am  not  familiar  at  all. 


a. 

Help  Screens 

b. 

_ On-1  ine 

c.  _ 

User-Friendly 

d.  _ Oata  Base  Management  System  (DBMS) 

e.  _ PRICES  (Price  History  Data  Base) 

f.  _ Mini -computer 


g.  _ Fourth  Generation  Language  (4GL) 

h.  _  DPCI  (Distributed  Processing  for  Contractual  Input) 

i.  _  Source  Oata  Automation  (AMIS  Forms) 


j.  _ Mainframe 

k.  _  FAR  On-line 

l.  _ Queries 


m.  _  String  queries 

n.  _  Natural  Language  commands 

o.  _ Automated  PNM  and  Pricing  System  (APPS) 


p.  _ Award  Fee  Tracking  System  (AFTS) 

q.  _  Integrated  Distributed  System 

r.  _  MILSCAP  (Military  Standard  Contract  Administration  Procedures) 


s.  _ Menus 

t.  Real-time 
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31.  Approximately  what  percent  (J)  of  your  work  week  is  involved  with 
data  entry  (e.g.  inputting  data,  resolving  input  problems,  etc.)? 

_ % 

32.  Approximately  what  percent  (1)  of  your  work  week  is  involved  with 
retrieving  data  (e.g.  queries,  reports,  etc.)? 

_  % 

33.  Approximately  what  percent  (i)  of  your  work  week  involves  using  output 
(e.g.  planning,  preparing  status  reports,  assessing  workloads,  etc.)? 

_ l 

34.  In  your  office,  who  primarily  inputs  data  into  PMS? 

a.  Clerks 

b.  Buyers 

c.  Procurement  Assistant 

d.  Other  _ 

35.  Where  is  the  closest  terminal  with  access  to  AMIS  located  in  relation  to 
your  desk? 

a.  Clerk' s  workstation 

b.  Centrally  located  "computer  workstation" 

c.  My  workstation 

d.  Other  _ 

36.  With  regard  to  your  answer  to  Question  35,  is  this  location  convenient 
for  you? 

a.  Yes 

b.  No 

c.  Other  _ 

QUESTIONS  37  -  43  ARE  BACKGROUND  QUESTIONS. 

37.  I  am  a: 


a. 

Contracting  Officer 

g- 

Group  leader 

b. 

Contracting  Officer  and  Buyer 

h. 

Branch  Chief 

c. 

Clerk 

i . 

Division  Chief 

d. 

Procuremert  Assistant 

j. 

Director 

e. 

Buyer/Negcti itor 

k. 

Other 

f.  Procurement  Analyst 
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38.  My  present  grade/rank  is: 


a.  If  Civilian  GS- _  or  GM- _ 

b.  If  Military,  Grade  E- _  or  (J-_ 

39.  What  type  of  contracting  do  you  do? 

a.  Research  and  Development  (RSD) 
o.  Systems 

c.  Both  R4D  and  Systems 

d.  Other 


40.  What  type  of  organization  do  you  support? 

a.  Systems  Program  Office  (SPO)  that  supports  one  major  weapon  system 

b.  Systems  Program  Office  that  is  responsible  for  a  variety  of  weapon 
systems  and/or  subsystems. 

c.  Research  and  Development  Laboratory 

d.  Test  Organization 

e.  Other  _ 

41.  My  cumulative  experience  in  a  Government  acquisition-related  job  is: 

a.  Less  than  one  year 

0.  At  least  one  year  out  less  than  three 

c.  At  least  three  years  but  less  than  five 

d.  At  least  five  years  but  less  than  ten 

e.  At  least  ten  years  but  less  then  fifteen 

f.  Fifteen  years  or  more 

42.  I  work  at 


a.  ASD 

b.  BSD 
C.  ESD 

d.  MSD 

e.  SSO 

f.  ESMC 


g.  WSMC 

h.  RADC 

i.  AFFTC 

j.  AFSTC 

k.  Other 


43.  My  highest  level  of  education  is: 

a.  High  School  Diploma  or  Equivalent 

b.  Associates  Degree 

c.  Bachelors  Degree 

d.  Masters  Degree 

e.  Doctoral  Degree 

f.  Other 
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44.  Please  take  this  opportunity  to  add  any  additional  suggestions  or 
comments. 

You  may  want  to  elaborate  on  the  best  features,  worst  features, 
mi ssing  features,  or  features  that  you  think  should  be  de'leted  from 
AMIS.  Vou  may  want  to  describe  an  ideal  workstation  in  terms  of  computer 
applications  and  tools. 

ALL  suggestions  and  comments  are  welcome! 


THANK  YOU  FOR  PARTICIPATING  IN  THIS  SURVEY! 

PLEASE  PUT  THE  QUESTIONNAIRE  IN  THE  RETURN  ENVELOPE  AND  PLACE  IT  IN  BASE 
DISTRIBUTION. 


60 


Figure  33.  PMS  Effectiveness  by  Function. 
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Figure  40.  Training  Useful  by  Education. 


69 


,%ee 


Mgt  by  Function. 


2 


Percent 


£gee 


Contracting  kMj 


Figure  44  . 
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Figure  47.  More  Remarks  Needed  by  Function. 
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Figure  48.  Mgt  Uses  PMS  by  Buying  Activity 
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Figure  52.  Buyers  Input  PMS  Data  by  Buying  Activity. 
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Fiaure  57.  Satisfied  with  AMIS  by  Buying  Activity. 
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Figure  62.  Adequate  Training  by  Buying  Activity 
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Figure  63.  Adequate  Training  by  Function. 
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Figure  66.  Notice  Sufficient  by  Buying  Activity. 
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Figure  67.  Notice  Sufficient  by  Function. 
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Figure  68.  Changes  Work  by  Buying  Activity. 
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Figure  73.  Management  Committed  by  Function 
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Management  Comments 


1.  I  would  like  to  see  historical  data  transformed  into 
graphic  charts  with  monthly  updates  thru  the  computer  base. 
Currently  this  has  to  be  done  manually. 

2.  The  WANG  "VS"  is  causing  problems  in  our  office.  We  get 
"kicked  out"  of  it  in  the  middle  of  a  document,  sometimes 
losing  a  portion  of  what  has  already  been  typed.  There  are 
too  many  people  for  too  few  lines  which  delays  getting  our 
work  into  the  computer  terminal. 

3.  The  primary  difficulties  with  PMS  are: 

(1)  the  wide  variation  in  interpretation  of  how  to  input 
data.  Because  of  the  dynamics  of  the  procurement  process 
there  can  be  a  wide  variance  in  what  a  computed  milestone 
means,  e.g.  "PROPOSAL  RECEIPT"  with  multiple  updates. 

(2)  the  standard  milestone  for  given  actions  don't  ever 
seem  to  match  with  the  kinds  of  milestone  experiences,  nor 
are  the  schedules  realistic.  This  requires  tailored 
networks  for  everything  -  very  time  consuming  if  you  have  a 
large  workload. 

(3)  reordering  milestones  by  date  is  very  disconcerting. 
The  milestone  should  remain  in  a  constant  order  even  if 
dates  are  out  of  sequence.  We  are  event,  not  date  oriented 
-  more  user  friendly  to  always  have  events  in  same  order, 
even  if  they  in  actuality  don’t  flow  in  the  same  sequence  in 
a  particular  action. 

(4)  the  various  reports  don't  seem  to  crossflow  changes 
in  data  to  stay  current.  If  buying  plan  data  changes  -  the 
changes  need  to  flow  to  all  reports:  data  such  as  $  values, 
buyer,  NTE  -  these  can  change  as  well  as  dates. 

Regarding  AMIS  (DPCI) ,  there  are  too  many  forms  of 
access  to  input/create  error  ridden  reports.  In  particular 
contract  closeout  records.  Consistently  data  on  $  values, 
etc  are  not  current.  It  is  not  clear  even  after  research, 
who  is  supposed  to  do  what  or  how. 

4.  I  have  never  accessed  AMIS  myself,  therefore,  several 
questions  I  could  not  answer.  I  would  like  to  see  AMIS 
become  more  integrated  with  the  buyers  daily  routine.  That 
would  require  easier  access  such  as  a  terminal  on  every 
buyers  desk  and  educational  training  concerning  capabilities 
of  the  system. 

5.  Worst . down-time  or  non-availability 
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Worst . DD3S0  input  should  flow  1,2,3 . 8;  not  in  AFLC 

checking  order . 

6.  More  training  is  needed  on  the  system  to  know  what  it 
can  do  now,  would  be  a  benefit  to  me.  I  would,  then  be 
able  to  answer  your  questions  without  feeling  inadequate.  I 
believe  that  AMIS  is  useful,  even  with  its  problems,  to  aid 
in  managing  the  workload  within  my  branch,  however,  with  the 
workload  ever  increasing,  time  to  receive  training  becomes  a 
problem  and  training  becomes  a  mute  point . 

7.  Buyers  and  PCOs  are  already  working  10-12  hours  per  day. 
Don’t  have  a  lot  of  time  to  learn  to  input  AMIS  data.  Done 
by  clerk.  If  there  is  a  time  savings  with  that  -  need  to 
teach  us  how  to  use  quickly. 

Needs  to  be  an  easy  way  to  call  up  unique  reports,  i.e.  John 
Doe  has  10  PRs  with  status  of  each  -  all  as  a  one  line  entry 
and  all  on  one  page  VERSUS  Buy  Plan  for  each  action.  Can't 
carry  all  of  the  volume  to  every  meeting  I  go  to. 

8.  Surveys  are  difficult  to  fill  out;  difficult  to  convey. 
Need  a  one-to-one  meeting  with  users  to  see  how  it  is  used 
in  actual  practice. 

Users  need  to  be  able  to  be  very  specific  on  delay  reasons 
for  missed  milestones.  In  the  environment  of  120  day  source 
selections  buyers  and  PCOs  need  to  be  able  to  account  for 
every  day . 

9.  The  AMIS  manual  hasn't  been  updated  in  a  number  of 
years.  Reality  of  system  differs  from  manual.  One 
reference  -  easy  to  follow,  block  by  block  -  is  needed. 
-Imaging  and  CBD  electronic  submission  are  exciting  ideas 
and  needed. 

-AMIS  tie-in  to  LOTUS  1-2-3  for  bar  graphs  (ref  27e)  would 
be  useful . 

-Also,  your  one  page  buyer  workload  at  a  glance  would  be 
useful;  add  as  PMS-N  report. 

10.  CAI  is  (needs  improvement) ,  It  would  be  better  to 
publish  that  information  in  a  book. 

-You  still  have  impossible  logic  in  PMS .  After  we  input 
data  and  save  it,  we  get  a  message  that  asks  if  we  want  to 
continue  to  update.  If  we  say  yes,  it  puts  us  back  where  we 
tried  to  leave.  But  if  we  say  no,  the  message  says  that  any 
inputs  that  we  made  will  be  lost.  Somehow,  the  data  gets 
accepted,  But  your  messages  don’t  make  sense. 

-We  need  to  be  able  to  correct  errors  in  the  system, 
especially  as  they  relate  to  UCAs .  A  buyer  in  a  different 
organization  entered  the  wrong  office  symbol  on  one  of 
his/her  change  orders,  and  we  could  never  get  the  error 
fixed.  As  a  result,  the  change  order  showed  up  on  our 
reports  (screwing  up  all  summary  data)  for  18  months. 
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-You  must  make  it  easier  to  move  buyer's  office  symbols,  and 
make  them  good  down  to  5  letters.  It  takes  an  act  of 
Congress  to  get  these  codes  changed,  and  in  the  meantime, 
summary  reports  are  inaccurate,  mistrusted,  and  consequently 
unused. 

-We  need  a  lot  more  information,  and  flexibility,  in  the 
"requirements"  part  of  the  system.  Nobody  at  XXX  will  admit 
to  knowing  anything  about  it. 

11.  You  have  a  very  difficult  job.  Keep  up  the  Good  Hard 
Work . 

12.  It  is  difficult  to  get  through  to  the  system! 

The  idea  of  having  two  passwords  is  dumb! 

It  will  be  harder  to  get  into  the  system. 

13.  The  inclusion  of  a  TQM  program  into  AMIS/PMS  database 
to  perform  more  statistical  calculations  for  each  milestone 
by  type  action  and  to  generate  complete  process  control 
charts  (X-R  Charts)  would  be  the  most  valuable  change  for  me 
to  improve  and  better  control  the  acquisition  process. 

14.  I  have  been  tasked  at  XXX  to  work  with  the  ACM  and  user 
organizations  to  improve  the  AMIS/PMS  process  and  procedures 
at  XXX.  The  single  most  damaging  drawback  at  XXX  is  the 
ability  to  control  the  age  of  an  action  Ci.e.  stop  age  with 
delay  codes,  enter  network  start  date  based  on  our  ability 
to  complete  an  action  (i.e.  proposal  receipt  technically 
accepted) ,  or  have  a  second  age  field  representing 
procurement  age  versus  acquisition  age  since  this  is  used  by 
PK  only  with  no  SPO  access)  the  later  is  the  most  desirable 
to  realize  the  potential  and  intent  of  the  system. 

-Secondly,  more  report  generation  flexibility  is  needed. 

Our  needs  and  desires  at  XXX  are  different  than  other 
product  divisions  who  also  have  unique  needs. 

-Third,  if  possible,  people  developing  and  changing  the 
system  should  visit  the  product  divisions  to  see  first  hand 
how  the  system  is  utilized. 

15.  Changes  can  be  made  but  it  is  SLOW! 

-Access  time  is  SLOW! 

-Terminology  is  not  acquisition  terminology  in  many  cases. 
-You  need  to  work  with  system  constantly  to  be  comfortable 
with  it  -  not  friendly. 

-you  seem  to  assume  everyone  is  computer  literate.  They  are 
not  . 

16.  Recommend  scrapping  AMIS.  Utilize  new  system  similar 
to  TURF  (being  used  at  certain  AFPROs)  with  certain 
modifications . 

17.  I  suggest  that  AFSC  help  XXX  obtain  work  stations  for 
all  buyers  and  managers. 
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-I’d  like  to  see  a  self  inspection  program  similar  to  the 
FAR  database  that  is  based  on  logic  strings  that  allow  us  to 
key  in  $,  type  of  contract,  method  of  contracting,  etc  and 
have  a  series  of  appropriate  checklist  questions  come  up 
tailored  to  the  particular  file  being  reviewed.  It  needs  to 
compile  the  discrepancies  as  they  are  input  and  prepare  a 
summary  document . 

-The  FAR  database  needs  to  be  improved  so  we  can  write 
construction,  base  level  services  and  supply  contracts. 

18  The  biggest  complaint  with  AMIS  from  the  start  has  been 
that  it  has  little  benefit  to  the  main  line  worker  -  the 
buyer.  The  WANG  computer  is  the  first  step  towards  a 
helping  tool  for  the  buyer  but  the  AMIS  manual  has  never 
been  changed  to  reflect  what  the  operator  sees  on  the  DPCI 
screen.  The  frustration  level  has  thus  increased  because 
there's  now  no  relevant  reference  instruction  to  consult. 
-AMIS/PMS  can  be  a  helpful  tool  to  the  worker  as  well  as 
management  but  solid,  steady,  available  training  that 
explains  not  only  the  what  but  the  why  and  the  how  to  use 
must  exist.  This  should  be  a  full  time  job  and  the  person 
must  have  proven  teaching  skills  as  well  as  contracting  and 
computer  knowledge.  Until  you  win  the  hearts  of  the  buyers, 
the  system  will  be  plagued  by  errors  and  bad  data. 

19.  The  contract  information  that  is  contained  in  AMIS  is 
very  valuable  however  we  have  had  trouble  updating  it  when 
it  is  incorrect.  For  example,  when  our  contract  face  values 
or  obligations  is  incorrect,  it  seems  that  no  one 
understands  how  to  change  it,  or  the  process  is  so 
cumbersome  that  no  one  wants  to  bother. 

-I  believe  that  there  are  certain  inputs  made  into  AMIS  by 
the  contract  administration  side  of  the  house.  It  is 
unclear  to  me  what  they  do  in  this  regard,  how  it  affects  my 
contracts,  and  how  I  can  understand  all  of  this.  For 
example,  I  have  some  contracts  that  have  been  terminated. 

I  have  issued  all  appropriate  mods  that  should  have  changed 
the  code  from  A  -  active  to  T-  terminated,  however,  the  code 
is  still  showing. 

My  AMIS  people  tell  me  there  is  something  that  must  be  input 
by  the  ACO  community  in  order  for  my  contract  status  to  be 
correct.  I  feel  helpless  because  I  can’t  get  this  resolved. 
Mostly,  it  is  a  lack  of  knowledge  problem. 

-Getting  into  the  systems  is  a  problem  for  us  right  now.  We 
are  going  through  the  new  password  process  and  we  have  been 
shut  down  because  no  one  understands  how  it  works . 

-BOTTOM  LINE  on  what  is  the  best  system  for  the  user:  make 
everything  as  user-friendly  and  menu-driven  as  possible. 
Contract  negotiators  have  too  many  other  things  to  learn 
about  and  keep  track  of  besides  trying  to  figure  out  how  to 
get  into  their  computer  and  how  to  get  out  and  put  in  the 
information . 
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20.  System  is  hard  to  access  for  an  occasional  user.  Far 
too  many  steps.  Make  it  simpler  for  managers.  System  off¬ 
line  too  much.  Too  many  delays  in  getting  out  contracts. 
Always  waiting  on  AMIS  for  something.  Too  many  bugs  in  the 
system . 

21.  PMS  is  very  helpful  in  terms  of  reporting  and  tracking. 
I  haven't  figured  out  how  AMIS  is  helping  us  in  the  buying 
office.  Rarely  do  I  pull  an  AMIS  query.  Information  is 
incomplete  or  incorrect.  The  time  involved  in  training, 
input,  ACM  reviews,  do  not  result  in  a  benefit. 

22.  Who  uses  the  output,  like  DD35Q  information?  ...  the 
perception  is  that  the  data  is  not  accurate  and  therefore 
not  usable. 

-When  will  the  SPOs  be  tied  into  it.  There  is  a  lot  of 
wasted  time  providing  the  same  information  in  different 
formats  to  the  SPOs  and  the  AC  community.  That  brings  to 
mind  (that?)  the  AC  community  should  be  tied  in  to  AMIS 
also ! 

-Since  I  was  thrown  my  first  AMIS  manual  in  1978  not  much 
has  changed  with  respect  to  training  and  use  of  AMIS/PMS. 
-Although  I  support  AMIS/PMS  in  theory,  in  practice  I  think 
we  are  often  fooling  ourselves.  If  anyone  cares  to  discuss 
this  more  I  can  be  reached  at  XXX  . 

23.  AMIS  was  developed  as  a  reporting  tool  for  AFSC 
management.  It  is  not  "user-friendly"  and  must  have  been 
developed  by  programmers  with  little  knowledge  of 
contracting.  It  has  little  utility  at  my  level. 

24 .  AMIS  should  move  toward  applying  industry  standard 
protocols.  It  should  also  allow  for  enhancements  as 
technology  changes.  We  should  not  be  held  captive  of  one 
technology/contractor,  i.e.,  terminate  WANG! 

-The  overnight  BV2  process  is  the  worst  feature  of  AMIS 
because  it  directly  impacts  productivity. 

-AMIS  access  is  also  a  problem.  My  office  was  unable  to 
access  AMIS  for  a  total  of  97  hours  in  July  alone! 

-In  my  office,  we  use  the  WANG  terminals  for  E-MAIL, 
AMIS/PMS,  only.  We  use  Apples  and  Z-248s  for  word 
processing,  graphics,  spreadsheets,  etc.  This  is  very 
inefficient . 

25.  AMIS  down  time  is  excessive!  No  APS  systems. 

26.  Milestones  are  totally  unrealistic  both  in  nature  and 
t  ime  . 

-System  needs  to  be  on-line  and  branches  should  have  access 
to  AMIS.  Too  much  time  is  spent  trying  to  get  minor 
corrections  or  administrative  mods  out  of  word  processing 
center . 
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27.  We  need  more  reliable  communication  lines  between  our 
point  and  yours . 

-Need  more  user-friendly  system 

-PKQ  people  are  always  ready  to  support  our  needs  and  answer 
questions . 


Contracting  Officer  Comments 


28.  Until  very  recently,  there  were  absolutely  no  people 
with  buying/contracting  experience  in  the  AMIS  office. 

29.  Real  language  manual! 

Up-to-date  training. 

User  friendly  programming. 

30.  AMIS  is  a  Dynamic  effort  that  I  believe  is  growing  with 
the  ever-changing  contracts  business .  I  was  an  AMIS 
monitor  in  the  1976-1979  time  frame  and  have  seen  good 
improvements  and  applications.  I  believe  in  the  system  but 
have  been  away  from  it  on  a  day-by-day  basis.  Keep  up  the 
good  work! 

31.  Scrap  the  entire  system  to  cut  losses.  Totally  revamp 
and  truly  modernize. 

32.  We  administer  contracts  on  XXX.  There  is  not  an  AMIS 
terminal  here.  The  procurement  assistant  drives  approx  25 
miles  to  use  the  terminal  at  XXX  for  the  entire  group. 

He/She  has  difficulty  inputting.  It  takes  several  days  to 
recover  from  an  error.  We  do  not  have  a  lot  of  AMIS  inputs, 
but  on  occasions  where  we  do  this  is  our  biggest  time 
waster.  We  are  not  a  typical  AMIS  user,  so  I  don’t  believe 
this  report  will  be  representative.  I  was  unable  to  respond 
very  well  as  I  do  not  personally  use  the  AMIS  terminal.  We 
fill  out  the  worksheets  and  give  (them)  to  the  procurement 
assistant . 

33.  AMIS/PMS  is  great  when  the  prompts  for  the  novice  user 
are  available.  Don't  make  changes  without  letting  us 
know! ! 1 

34.  Frankly,  as  a  Contracting  Officer/Buyer  AMIS  does 
little  or  nothing  for  me  since  I  know  the  status  of  my  work. 
I  don't  need  an  AMIS  printout  to  tell  me  where  I’m  at.  I’ve 
always  viewed  AMIS  as  one  of  those  hassles  I  have  to  put  up 
with  to  accomplish  my  job.  Further,  the  AMIS/PMS  has 
further  complicated  my  job  because  it  demands  dual  inputs  — 
one  for  AMIS  and  one  for  the  local  contract  tracking  system. 
-However,  it  is  my  understanding  that  XXX  will,  in  the 
foreseeable  future,  have  the  ADPE  available  on  every  buyers 
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desk  to  facilitate  buyer  input  into  AMIS  and  further  buyers 
will  be  compelled  to  input  directly  from  their  desk  terminal 
to  AMIS.  So... I  would  hope  that  AMIS  will  be  simplified  as 
much  as  possible  and  oriented  to  serve  buyers  needs  as  well 
as  management  needs . 

35.  Management  information  about  contracting  actions  should 
be  generated  as  a  consequence  of  the  process  used  to  get  the 
job  done.  The  buyers  use  of  the  computer  should  be  a  tool 
to  generate  products  he/she  needs,  i.e.,  the  RFP,  file 
documents,  etc.  Then  tracking  data  for  mgt  purposes  should 
be  generated  as  a  result  of  using  the  tool.  Now  PMS  serves 
no  value  to  buyer  -  it  is  an  impediment  to  the  primary 
activity  -  issuing  RFPs  and  awarding  contracts .  A  special 
effort  is  required  to  input  data  strictly  for  management 
review.  Most  buyers/PCOs  in  systems  contracting  can 
effectively  track  their  workload  manually  because  they 
handle  a  small  number  of  actions  at  one  time.  In  fact,  at 
XXX,  we  keep  contracting  Directors  advised  of  status  on  our 
programs  by  manually  marking  up  a  word  processing  output 
listing  our  programs.  Clerks  then  update  this  data  base. 

PMS  seems  to  have  no  value  to  management  at  this  level 
either,  because  it  is  never  "fresh"  enough  and  big  programs 
are  constantly  discussed  -  so  the  "real  story"  is  in  a 
manual  report.  We  have  lots  of  micro  information  in  PMS, 
but  I  believe  only  the  macro  information  is  ever  really 
considered  by  anyone  -  because  if  you  really  want  to  know 
the  status,  you  call  the  PCO.  PMS  should  be  reduced  in 
scope  so  that  only  significant  milestones  are  tracked  -  and 
those  milestones  should  be  monitored  as  a  natural  outcome  of 
the  process  -  e.g.  A  milestone  is  noted  when  the  RFP  is 
generated  by  DPCI ,  when  RFP  is  mailed,  the  fact  is  input  be 

a  clerk  who  scans  the  cover  with  a  bar  code  reader,  etc  - 
AFLC  has  more  of  a  production  and  volume  oriented  system  - 
but  I  believe  they  have  some  good  ideas  -  if  not  familiar 
with  their  system,  you  should  contact  HQ  AFLC/PML  at  Hill 
AFB,  UT  to  get  some  ideas  which  could  perhaps  be  adapted  to 
AFSC .  But  whatever  you  do  -  you  should  make  PMS  user 
friendly  —  get  rid  of  nonsense  codes  and  numbers  needed  to 
input  data  -  make  the  whole  thing  menu  driven  with  questions 
keying  the  next  step  -  and  by  all  means  -  make  it  easier  to 
log  onto.  One  should  not  need  3  userids  and  passwords  to 
get  from  WANG  office  into  PMS.  Good  luckl  You  have  quite  a 
challenge  ahead  of  youl 

36.  At  XXX  the  AMIS  system  is  "down"  far  too  frequently. 

New  equipment  is  desperately  needed  -  I  do  not  understand 
why  this  is  not  a  top  priority  item. 

-System  is  not  user  friendly  and  is  too  slow. 

-Lack  of  sufficient  terminals  hampers  employee  utilization 
of  many  of  the  system’s  features. 
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37.  The  AMIS/PMS  systems  generate  more  work  than  they  save. 
It  appears  that  these  systems  were  designed  in  a  vacuum. 

The  people  who  are  required  to  input  to  the  systems  and  use 
them  on  a  daily  basis  were  never  asked  for  inputs  into  their 
designs.  It  appears  that  people  apart  from  and  unconcerned 
for  the  user  designed  these  systems.  The  system  provides  no 
usable  products  for  the  users;  delays  procurements;  requires 
manual  backup;  and,  most  likely  provides  erroneous 
information  to  management.  I  am  glad  that  AMIS  does  not 
generate  my  paycheck.  Aren’t  you? 

38.  The  system  has  many  problems  such  as: 

-Basic  350  does  not  flow  to  mods 

-The  system  is  down  quite  a  lot 

-The  system  itself  is  a  major  block  in  getting  documents 
prepared  and  distributed. 

-Without  system  as  PCO  could  get  mods  typed  in  half  an  hour 
and  out  the  door. 

-How  is  the  system  supposed  to  be  better?? 

39.  Train  all  levels  of  personnel  at  XXX  -  no  one  really 
understands  the  system  or  how  to  use  it. 

40.  In  order  to  make  the  system  a  tool  that  is  useful;  it 
must  be  recognized  that  all  the  buyers  and  contracting 
officers  must  have  an  AMIS  terminal  dedicated  for  his/her 
use.  At  present  AMIS  is  a  reasonably  good  tool  for  senior 
management  to  query  with  regard  to  status  of  PR  packages  but 
of  little  use  to  the  buyers  in  the  pits.  To  make  it  a  tool 
that  would  be  useful  to  the  working  level  buyer  the  on-line 
capability  must  be  sold  to  show  us  how  this  management 
system  will  benefit  us  and  not  just  make  management  more 
distant  from  the  individuals  doing  the  work.  We  must  have 
the  on-line  contract  drafting  capability  that  includes  all 
contract  clauses,  on-line  FAR  research  capability,  plus  all 
the  other  capabilities  listed  in  question  20  of  this  form. 

41.  WANG  downtime  is  too  high. 

-Buyers  are  not  allowed  access  to  AMIS/PMS. 

-Updates  to  data  base  are  not  current . 

-PKO  will  not  decentralize  PMS  tracking. 

-No  access  to  other  features . 

-DD  350s  have  to  input  same  info  over  and  over. 

-WANG  not  compatible  with  rest  of  base  computer  system. 

-Use  word  processing  most. 
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42.  As  a  contract  negotiator  with  a  heavy  work  load,  there 
is  little  time  to  halt  the  work  and  practice  on  a  computer. 
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True,  it  is  useful  and  needed  for  some  particular  tasks. 
However,  they  do  not  occur  often  enough  to  gain  any  real 
proficiency.  Then,  when  one  does  have  a  task  to  perform,  he 
does  not  have  the  skill  to  do  it  proficiently.  I  have 
acquired  some  proficiency  on  one  system,  then  they  come  in 
and  change  it,  forcing  one  to  start  all  over  again.  Don’t 
have  the  time  for  that.  The  faulty  system  is  now  in  place 
and  I'm  too  behind.  I  have  never  seen  a  computer  negotiate 
with  a  contractor. 

43.  PMS  works  great  for  me  to  manage  my  workload  with  a 
quick  look.  However,  management  relies  on  this  report  to 
determine  my  milestone  status  and  that  info  is  not  up  to 
date  nor  is  it  always  accurate  due  to  system  problems  and 
input  inconsistencies. 

44.  Please  provide  course  to  clerks  on  AMIS  (Contract 
Writing) . 

Provide  AMS/AMIS  courses  to  buyers. 

45.  It  is  very  inconvenient  to  use  the  workstation  because 
we  only  have  3  for  15  people. 

46.  PMS  is  a  good  guide  or  "quick  look"  type  system  -  but 
it  is  not  an  adequate  system  to  determine  workload  or 
micromanage  (management  complains  because  updates  are  not 
being  made  or  milestones  are  passed  before  they  are  updated. 
We  have  real  work  to  do)  . 

47.  Myself  the  word  computer  or  workstation  frightens  me. 

I  do  not  understand  computers  and  I  do  not  use  them.  I  have 
been  in  the  procurement  business  for  over  20  years.  I  think 
the  PMS  system  and  AMIS  system  is  necessary.  It  can  and 
must  be  made  effective  to  improve  my  job  and  management 
capabilities.  BUT  train  me  in  the  basic  use  of  a  computer 
to  help  me  to  improve  my  job  and  I  will  be  happy  to  use  both 
systems . 

48.  I  have  found  AMIS  to  be  complex  and  difficult  to  use  - 
definitely  not  user  friendly.  As  a  buyer  it  has  only 
complicated  my  job,  creating  redundancies  and  complications 
which  only  make  the  work  more  time  consuming.  More  training 
might  help,  but  a  machine  which  only  regurgitates  input  is 
of  no  practical  use  to  me.  I  need  a  decent  word  processor 
more  than  I  need  AMIS.  Thanks  for  asking  though  1 

49.  Biggest  problem  with  AMIS/PMS  is  inaccurate  data. 
-Management  does  not  take  full  advantage  of  available 
tailored  reports. 

-Use  of  system  is  limited  to  specific  persons  in  the 
organization  who  are  not  always  available  or  amicable  when  a 
report  is  needed  by  lower  grade  acquisition  personnel. 
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50.  The  worst  feature  of  AMIS  is  management  constantly 
emphasizing  need  for  all  personnel  to  do  their  own  data  when 
I  only  do  some  of  the  data  on  a  very  sporadic  basis  and  have 
to  re-learn  how  each  use.  Some  things  seem  better  suited  to 
have  central  point  where  collectively  entries  are  done  often 
enough  for  person(?)  to  know  how.  Also,  as  noted 
previously,  I  think  it's  bad  for  numerous  people  to  be 
spending  lots  of  time  to  get  into  the  system  instead  of  a 
central  point . 

51.  Immediately:  add  a  prompt  that  says: 

"TO  SAVE  THIS  INPUT  PRESS  Z"  where  necessary. 

-Program  menus  to  pop  into  place  instead  of  slowly  scrolling 
down;  make  all  menus  more  accessible  (like  Apple  and  MAC) 
-But  my  most  valuable  comment  is  this  -  reverse  the  input 
process.  Let  us  type  our  data  in  word-processing  format  as 
required,  then  use  NEW  technology  to  SCAN  that  document  and 
thus  input  the  data  there  on  directly  into  machine/computer 
language.  This  technology  is  relatively  cheap  and  available 
in  every  Radio  Shack/computer  store  in  this  area.  Cost  to 
input  the  typed  page  is  about  $.06  at  commercial  rates. 
Machine  to  input  (a  scanner)  is  about  $1000. 

52.  The  AMIS  system  is  by  far  an  antiqued  computer  by 
modern  standards.  The  usefulness  of  the  system  is  minimal. 
In  addition,  a  persons  contracting,  negotiation  and  contract 
administration  is  not  judged  by  contract  experience  rather 
by  AMIS  input  and  knowledge.  The  basic  theme  at  this 
facility  is  "If  you're  a  good  AMIS  clerk  you  must  be  a  good 
contracting  person."  What  is  seen  is  overpriced  clerks 
(that  is  GS-11 , 12 , 13' s  inputting)  do  AMIS  work. 

53.  One  person  in  our  organization  is  labeled  "GURU"  and  we 
get  whatever  info  he/she  deems  relevant  or  necessary,  and 
it's  usually  not  in  a  timely  manner.  Many  discrepancies 
between  AMIS  and  Small  business  regarding  what  info  will 
reject.  Too  much  second  guessing  with  AMIS.  Tired  of 
hearing,  "AMIS  won't  allow  you  to  input  that!" 

54.  BSD  was  using  a  direct  input  via  Z-248.  It’s  now  using 
a  local  VS-100  to  control  data  input.  This  system  is  very 
trying.  Reports,  outputs,  and  DPCI  require  exiting  PMS  and 
using  WANG  Wordprocessing  to  edit  and/or  print  the  output. 
Prior  to  this  the  buyer  could  get  outputs  at  a  much  quicker 
rate.  This  system  requires  2  times  the  effort  to  get  a 
report.  This  ties  the  clerks  more  and  more  to  the  computer 
and  the  required  time  to  move  a  document  thru  AMIS  has 
doubled.  The  system  is  would  around  WANG  equipment  too 
tight  which  requires  the  user  to  be  familiar  with  numerous 
word  processors  all  conflicting  with  the  local  area  network 
programs.  Messages  that  require  distribution  must  be 
retyped  in  WANG  to  be  used  on  AMIS  and  WANG  messages  can  not 
be  sent  on  the  LAN. 
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55.  My  suggestion  is  to  have  AMIS  keep  an  entire  contract 
on  computer  file  (hard  drive) ;  then  whenever  a  change  is 
made,  the  computer  can  prompt  you  to  tell  you  where  all  the 
changes  are  needed  throughout  the  entire  contract. 

56.  AMIS  does  not  work  for  us,  usually  against  us.  It's 
not  flexible  when  something  is  needed  in  a  hurry;  system  is 
down  or  you  can't  get  in. 

57.  Fact  -  not  all  DCAS  ACOs ,  buyers,  CAs ,  FOs ,  etc  use 
AMIS. 

Example:  An  "A(admin)  mod"  with  a  price  increase.  No  AMIS, 
therefore  for  each  A  mod,  we  have  to  cut  another  mod  to  get 
new  face  value,  obligation  amount,  etc.  info  into  AMIS. 

-I  use  both  AMIS  and  PMS  as  a  buyer.  I  like  both.  I'm  told 
that  this  makes  me  weird!  KEEP  UP  THE  GOOD  WORK! 

58.  All  input  to  AMIS  and  PMS  is  done  by  XXXX .  Buyers, 
etc,  do  not  have  direct  input  capability.  Two  summary 
sheets  that  include  the  current  status  on  all  of  my  actions 
are  the  most  useful  tool  that  I  have,  not  the  separate  PMS 
sheets.  Separate  PMS  sheets  become  an  extra  burden  because 
I  have  to  update  them  in  addition  to  my  summary  sheets;  I 
must  turn  them  in  to  XXXX  and  take  time  to  refile  them  each 
time  there  is  a  change.  My  summary  sheets  provide  easy 
total  workload  management  and  I  only  have  to  input  to  WANG 
on  a  weekly  basis.  I  can  quickly  pencil  in  notes  during  the 
week . 

59.  I  am  new  so  please  note  that.  I  have  been  at  XXX  less 
than  2  months.  However,  I  have  received  no  training  in 
regard  to  AMIS  or  document  prep. 

60.  AMIS  is  not  flexible  enough. 

61  . 

A.  I  believe  the  worst  features  of  AMIS  are: 

-amount  of  computer  down-time 

-contracting  officer’s  lack  of  concern  as  to  AMIS 
problems  (BV2,etc)  and  their  "arbitrary"  waivers  of 
AMIS  requirements 

-no  effective  way  to  interface  AMIS  with  secondary 
delegations  (mods)  and  delegated  contracts 

B.  The  best  features: 

-purchase  request  tracking 

-verification  method  for  uniform  contract  formatting 

62.  PMS  is  useful  for  developing  schedules  for  planning 
purposes  for  a  new  contract.  It  is  also  helpful  for  keeping 
track  of  lengthy  (time-wise)  supplemental  agreements. 
However,  it  serves  no  useful  purpose  to  Buyers  for  Admin 
Mods  or  Supplemental  Agreements  which  do  not  take  long  to 
put  on  contract  . 
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-Workload  should  not  be  judged  entirely  by  entries  into  PMS. 
The  amount  and  extent  of  additional  duties,  correspondence, 
etc  should  also  be  taken  into  consideration. 

-The  automated  milestones  for  a  new  source  selection  do  not 
meet  XXX' s  streamlined  Acquisition  Goals.  It  would  be 
useful  to  have  an  automated  funding  summary. 

63.  Updating  is  cumbersome  when  you  have  to  keep  accessing 
the  menu  to  get  back  to  a  particular  file. 

-since  the  DD350  is  in  the  computer,  having  the  1547/1861 
and  1279  report  computerized  would  be  helpful. 

-Inputting  an  actual  date  means  regenerating  schedule  dates 
back  to  office  predictions;  not  machine  predictions  (very 
time  consuming) . 

64.  The  system  is  much  better  than  it  was  when  I  first 
started  using  it  in  1987.  It  still  takes  a  long  time  for  a 
new  person  to  learn  the  ins-and-outs  of  using  PMS. 

-One  of  the  big  frustrations  is  the  lack  of  the  systems 
ability  to  remember  the  action  you  are  working  on.  Tor 
example,  if  you  create  a  PMS  Buying  Plan  for  a  contract 
action,  you  have  to  exit  out  of  the  Buying  Plan  to  get  into 
the  DD350  mode  and  then  you  have  to  retype  the  contract 
number.  It  would  be  much  more  efficient  to  enter  the 
contract  number  first  and  then  be  able  to  move  from  Buying 
Plans  to  DD350s  in  a  simple  keystroke. 

65.  I  think  more  training  for  those  who  use  AMIS/PMS  is 
called  for.  But  most  important,  we  need  to  know  what 
individual  within  each  organization  is  responsible  for  input 
procedures.  As  a  buyer/negotiator  I  have  often  found  that 
our  clerks  and  PAs  put  all  AMIS/PMS  duties  "on  the  back 
burner."  I  feel  that  if  this  data  entry  of  all  AMIS/PMS  is 
in  your  PD,  then  it  is  that  individuals  responsibility. 

Most  buyers  do  not  have  the  time  and/or  knowledge  to  perform 
all  AMIS/PMS/DPCI  tasks. 

-In  most  SPOs ,  buyers  do  AMIS/PMS/DPCI  themselves.  However, 
I  feel  this  takes  time  away  from  my  more  important  tasks. 

66.  You  know,  if  you  people  would  just  leave  things  alone 
for  a  while  and  let  the  work  force  become  knowledgeable  in 
some  of  these  areas  which  seem  to  "constantly"  be  in  a  state 
of  flux;  maybe  morale  would  improve,  productivity  would 
improve,  etc.  As  it  is,  most  contract  negotiators  don't 
bother  trying  to  fully  learn  or  understand  all  these  changes 
because  most  of  us  figure  the  personnel  at  the  upper 
echelons  (most  of  whom  have  little  experience  in  buying  or 
it  has  been  years  since  they  were  in  a  buying  activity)  will 
change  it  anyway. 

67.  Make  getting  on  system  and  logging  on  much  simpler  and 
timely . 
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68.  I  have  no  idea  who  uses  AMIS  or  why.  PMS  is  used  by 
management  to  track  the  status  of  contract  actions  but 
nothing  else. 

69.  Inputting  a  DD350  is  definitely  not  user  friendly.  I 
have  spent  at  times  over  8  hours  with  a  clerk  to  help  her 
get  the  350  into  the  computer  correctly. 

70.  I  find  AMIS  abysmal  and  frustrating.  I  understand  the 
need  to  have  a  central  computer  reporting  and  tracking 
system  and  agree  the  theory  and  promise  of  possible  benefits 
make  the  system  very  desirable.  However,  the  system  is 
extremely  difficult  to  use.  Access  to  the  system  at 
Wright-Patterson  can  keep  tied  to  a  terminal  all  morning 
long  and  accomplishing  nothing  constructive,  however  at 
times  the  information  must  be  input.  The  system  is 
extremely  difficult  to  learn.  After  working  with  it  awhile 
it  is  not  to  bad,  but  to  try  to  learn  something  new  is 
usually  such  an  annoying  process  it  is  not  worth  the  effort. 
The  messages  of  how  to  clear  up  problems  are  far  to  brief 
and  abbreviated  to  be  of  any  use.  The  interaction  with 
other  data  bases  such  as  DPCI  cause  problems.  I  have  had 
actions  held  up  to  a  week  on  several  occasions  when  the 
modification  is  correct  but  it  does  not  agree  with  the  data 
base.  I  have  tried  to  use  the  AMIS  user  manual  and  find  it 
to  be  one  of  the  worst  manuals  I  have  ever  seen.  A  more 
direct  or  reasonable  manual  combined  with  access  to  the 
system  would  do  more  to  further  use  of  the  AMIS  system  than 
new  features  or  capabilities. 

71.  As  you  may  be  able  to  tell,  most  of  my  AMIS  experience 
is  with  PMS.  This  office  thinks  the  PMS  system  is  aptly 
named.  Our  biggest  gripe  is  the  trouble  in  printing  out  our 
status  on  each  action.  Using  the  shift  and  print  screen 
keys  gives  us  very  jumbled  DD350s  -  the  PMS  buyer's 
milestone  display  is  not  quite  so  bad. 

-My  other  comment  is  that  I  have  a  very  odd  contractor  who 
takes  from  4-12  months  to  respond  to  an  RFP,  yet  the 
director  wants  the  milestone  display  entered  as  soon  as  the 
RFP  (for  mods)  goes  out.  Several  have  eventually  been 
canceled  because  we  could  not  get  a  response.  When  I  try  to 
re-use  those  mod  numbers,  I  can't.  Now  what?  There  are  no 
mods  against  the  P0Q0Q0#’sl 

72.  I  am  a  new  user  of  AMIS/PMS  and  therefore  do  not  know 
enough  about  the  system  to  comment  at  this  time. 

73.  AMIS  is  a  very  useful  tool  for  planning  purposes.  The 
perspective  must  be  maintained  that  this  is  a  tool  to  be 
used  and  is  not  a  master  which  dictates  how  planning  is 
done.  Therefore,  time  and  resources  devoted  to  the 
mechanics  (inputs,  queries,  etc)  must  be  commensurate  with 
the  value  as  a  tool. 
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74.  Make  the  outputs  from  an  AMIS  "string"  clearer. 

Sometime  when  you  pull  a  string  (i.e.  tracking  funding)  you 
have  to  make  some  assumptions  as  to  what  the  info  you  have 
been  given  means .  The  legends/descriptions  are  not  always 
clear . 

75.  The  PMS  system  could  be  an  effective  management  tool  if 
used  properly.  I'm  not  convinced  its  happening  in  many 
cases.  Inputting  and  updating  of  data  is  very  time 
consuming  and  not  getting  feedback  from  immediate 
supervisors  seems  to  be  defeating  the  purpose. 

-As  for  AMIS.  Why  aren't  there  more  dedicated  AMIS  lines? 
One  line  per  office  becomes  a  logistic  nightmare. 

-An  ideal  workstation  is  one  where  every  two  buyers  have 
access  to  a  dedicated  AMIS/PMS  line. 

76.  PMS  needs  a  lot  of  improvement.  It  is  not  user 
friendly  and  is  a  "pain"  to  use.  Buyers  hate  it  because  it 
forces  us  to  give  input  with  a  lot  of  "pad"  in  it  just  to 
get  the  system  off  of  our  backs.  If  you  continue  to  change 
dates  for  milestones,  it  appears  as  though  you're  not  doing 
your  job.  But,  the  fact  of  the  matter  is  that  it  is 
difficult  to  make  predictions  because  there  are  so  many 
people  involved  with  your  acquisitions.  The  system  doesn't 
always  allow  you  to  explain  yourself.  We  are  sometimes 
pushed  into  putting  in  any  date(s)  just  so  someone  can 
complete  a  report.  And  then  one-half  of  the  time  PMS  fights 
you  while  you're  trying  to  input.  It's  a  waste  of  time.  We 
did  just  fine  before  we  had  PMS. 

77.  I  believe  that  PMS  is  extremely  user-friendly, 
cumbersome  and  frustrating.  Entry  procedures  are  changed 
frequently  and  our  office  is  not  notified  until  we  find  we 
can’t  access  the  system.  Rainy  weather  plays  havoc  with  the 
system.  To  update  a  Buy  Plan  you  do  not  have  access  at  the 
time  you  are  updating.  MU  does  not  show  what  you  are  doing 
until  the  action  is  complete.  When  you  try  to  exit  a 
command  the  user  is  asked  "Do  you  want  to  continue  updating? 
Yes  or  No"  This  is  an  unnecessary  step  and  slows  the 
process.  In  response  to  the  first  question  on  this 
questionnaire  -  PMS  is  an  effective  tool  for  tracking  the 
overall  status  of  contract  actions  -  It  could  be  more 
effective  if  the  buyer  was  able  to  input  actual  dates  prior 
to  PR  input  date.  There  are  times  when  an  action  is  worked 
prior  to  the  issuance  of  a  PR.  I  try  to  put  the  information 
in  "Remarks  or  Delay  Reason"  but  it  would  be  helpful  if 
under  "Actual"  you  could  input  the  Actual  date  of  action. 
The  system  is  very  slow  and  requires  a  great  deal  of  a 
buyer's  time.  In  addition,  many  buyers  are  trying  to  access 
one  computer.  This  impedes  proper  usage. 

78.  Milestones  do  not  always  match  up  with  the  type  of 
contract  and  the  dollar  thresholds. 
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-Limits  are  too  stringent  regarding  the  changing/deleting  of 
forecast  and  actual  dates . 

-Suggest  running  the  report  every  other  week  instead  of 
every  week, 

-If  buyers  were  able  to  do  their  own  inputting,  errors  and 
problems  could  be  corrected  right  away. 

-Allow  more  space  for  remarks  and  reasons  for  delays. 

79.  Concept  of  the  system  is  good  and  it  seems  fairly  easy 
to  use. 

-Worst  feature  is  the  overnight  reports.  Should  be  able  to 
get  these  reports  in  a  quick  time  frame,  i.e.  BV2s . 

-Since  I  am  at  an  off  base  location  the  system  seems  to  be 
down  quite  a  bit  due  to  either  the  modem  to  the  base,  or  at 
the  base  itself,  or  at  AFSC . 

80.  We  input  into  AMIS  on  a  weekly  basis  to  give  status  to 
managers.  Obviously,  it  isn't  a  useful  tool  for  them 
because  there  are  several  status  sheets  generated  each  week, 
requiring  the  same  information  that  we  put  in  PMS . 

-Buyers  spend  more  time  providing  status  than  giving 
attention  to  PRs  and  awarding  contracts. 

81.  Only  YYY  has  access  to  AMIS/PMS  at  XXX.  This  is 
unacceptable  since  buyers/CO  are  responsible  for  the 
informati-m  but  have  no  control! 

-Lack  of  computer  knowledgeable  individuals  is  high.  How 
can  AMIS  be  effective/efficient  if  management  is  not 
computer  smart?" 

-Present  system  is  ineffective/inefficient  to  buyers/COs 
needs  due  to  mismanagement  on  the  part  of  YYY  at  XXX!! 
-Computer  system  is  unreliable  (high  downtime) .  I  believe 
this  is  based  on  lack  of  actual  computer  knowledgeable 
individuals.  System  at  XXX  is  run  by  individual  who  doesn't 
know  what  he/she  is  doing  and  will  not  release  any  control. 
-Lack  of  computer  knowledge  on  the  part  of  all  management 
makes  system  ineffective  (do  not  know  or  understand  the 
system) . 

-Why  must  DD  350s  be  processed  for  actions  under  DFARS 
thresholds . 

-If  AMIS  contains  information  (DD  350)  from  previous  actions 
on  the  same  contract,  why  must  paperwork  be  filled  out 
again?  Almost  all  information  is  already  input  into  the 
system;  can  it  not  compile  it  into  a  DD350?  This  takes 
buyer  time  to  reaccomplish  every  time. 

82.  AMIS  system  should  be  expanded  to  allow  more  on-line 
access . 

-Down-time  is  too  high. 

-Updates  to  data  base  are  not  kept  current . 

-Why  are  DD350s  under  $25,000  required? 

-Information  required  for  DD350s  need  to  be  input  every  time 
DD  350  required  for  same  contract. 
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-Best  feature  is  the  word  processing. 

83.  More  reliance  is  being  placed  on  buyers  to  make  data 
entries  instead  of  relying  on  clerks.  This  is  a  good  idea, 
if  enough  terminals  are  available,  but  are  not  -  presently 
buyers  are  wasting  their  time  on  standby. 


Procurement  Clerk  Comments 


84.  Input  of  data  in  AMIS/PMS  is  repetitive.  You  have  to 
go  through  many  steps  to  do  something  fairly  simple. 
Input/retrieval  could  be  much  simpler  if  a  new  input  system 
were  created. 

85.  DD350  -  Believe  input  would  be  simpler  if  information 
was  pulled  from  last  mod  and  not  the  basic  contract , 
Sometimes  codes,  names,  addresses,  etc.,  change  after  basic 
is  written.  This  way  information  would  have  to  be  input  in 
only  one  DD350  and  not  each  time  you  do  a  350  for  a  new  mod. 

86.  I  believe  that  AMIS  has  good  features,  there  just  need 
to  be  more  access  to  phone  lines  for  hook-up.  There  are  2 
computers  with  AMIS  capabilities  in  my  office  but  only  one 
phone  line  to  hook  up  to  . 

-AMIS  orientation  training  would  be  useful  to  all  newcomers. 
-It  would  be  very  helpful  if  AMIS/PMS  was  accessible  over 
the  weekends  on  a  continuous  basis.  Some  weekends  the 
system  is  up  on  Saturdays  (but  one  can  not  depend  upon  it 
always  being  available) . 

87.  The  PMS  system  was  an  easy  and  user  friendly  system  - 
Until  the  input  of  DPCI .  DPCI  has  taken  a  one  step  process 
and  made  it  a  5  step  process.  DPCI  was  input  and  CPTs 
removed  without  any  concern  for  how  to  transfer  from  CPT 
format  to  DPCI.  DPCI  BV2s  reveal  errors  (lack  of  info  on 
68X)  but  when  trying  to  correct  errors  it  is  found  that  no 
such  "slot"  is  available  for  the  needed  info.  To  transmit 
documents  through  DPCI  many  of  the  pages  must  be  manually 
"rigged"  in  COM  DOCUMENT  status  before  transmission  -  Do  to 
not  being  able  to  input  the  proper  info  When  in  the  DPCI 
format.  DPCI  was,  and  still  is,  the  biggest  waste  of  money 
and  time  that  this  organization  could  have  gotten.  Putting 
it  nicely  DPCI  (needs  improvement) . 

88.  I  wish  that  the  buyers  could  do  their  own  buy  plans. 

I  feel  that  DPCI  will  not  be  used  by  95%  of  the  buyers  and 
PCOs  . 

Buyers  have  an  attitude  problem  when  it  comes  to  learning 
how  to  do  some  of  their  own  work  and  most  of  them  can't  even 
operate  computers. 
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89.  I  feel  they  change  the  ways  of  doing  things  far  too 
often.  You  learn  one  system  and  they  change  it.  This  is 
very  frustrating. 

90.  I  think  that  AMIS/DPCI  Program  is  very  cumbersome. 

Every  time  I  do  a  mod  in  DPCI  there  are  just  too  many  steps 
required  to  input,  send,  print-out  and  validate.  I  feel  the 
input  of  mods  should  be  done  directly  on-line  like  the 
DD350s  and  Buy  Plans  are  done. 

91.  Request  more  training  on  AMIS. 

-When  using  DPCI  (AMIS)  why  can't  the  system  create  a 
communication  document  and  a  print  document  for  the  user? 
-Why  can't  an  option  be  added  to  the  system  that  will  allow 
the  user  to  copy  text  to  disk  without  having  to  archive  to 
disk? 

-Why  is  it  so  hard  to  make  a  change/correction?  The  system 
should  be  able  to  allow  the  user  to  specify  the  screen  they 
would  like  to  go  to  without  going  through  every  screen. 

92.  When  we  do  a  DD10S7  on  AMIS,  we're  asked  different 
questions  every  time  we  do  one.  We  need  some  uniformity  on 
the  questions.  We  have  tried  to  create  a  worksheet  the 
buyers  can  use  to  do  a  DD1057  but  they  always  seem  so  be 
missing  about  5  questions  -  and  they're  always  different 
questions  -  so  we  have  no  uniformity  as  to  what  to  expect 
and  what  to  ask  the  buyers  for  . 

93.  Best  Feature:  Overall,  it's  great  for  all  intended 
purposes 

Worst  Feature:  Too  many  data  entries  for  an  end  result  and 
sometimes  inconsistent  with  what  or  how  those  entries  are 
input  (too  many  processes) . 

Missing  Features:  On-line  capability  (although  I  would 
guess  that  depends  on  what  type  computer  one  has) 

Ideal  Workstation:  That  everyone  (PCOs ,  Buyers , Clerks)  have 
a  computer  (PC)  on  their  desk  with  on-line  capability. 

Unless  this  happens,  Contracting  automation  can  never  truly 
be  understood  or  appreciated  for  which  it’s  intended.  To 
retain  proficiency,  one  needs  daily  access. 


Procurement  Assistant  Comments 


94.  Until  recently  I  had  direct  access  to  AMIS  through  a 
FALCO.  This  was  taken  away  from  me  and  now  I  must  use  my 
Z-248,  enter  the  WANG  system,  then  dial  AMIS.  It  is  very 
slow  and  time  consuming.  If  I  have  a  phone  inquiry  about  a 
contract,  now  I  must  end  whatever  I'm  doing  and  go  through 
all  those  steps  to  access  AMIS.  I'd  prefer  one  system 
dedicated  to  AMIS  with  quick  access . 
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I  found  many  errors  in  AMIS  and  trying  to  correct  them 
is  a  monstrous  task.  Wish  we  had  direct  access  to  AMIS  SPO 
personnel  to  help. 

DCASMAs  should  use  AMIS  to  track  contract  closeouts  and 
input  info  promptly.  It  would  save  hours  trying  to  check  on 
status  of  old  contracts  if  we  could  just  use  (the)  computer 
instead  of  repeated  letters  and  phone  calls. 

95.  Worst  features:  Wasted  time  in  "down  time"  and  waiting 
for  system  to  be  "up." 

96.  A  problem  I  have  with  AMIS  is  in  pulling  reports  from 
PMS .  If  the  machine  gets  blocked  or  knocks  you  off  line  you 
can’t  tell  it  to  start  at  a  certain  page.  If  it’s  a  100 
page  report  and  you  were  on  page  60  when  something  happened 
you  have  to  start  all  over  again. 

97.  I  do  not  access  PMS  AMIS  in  anyway. 

98.  Could  be  a  more  effective  tool  if  the  buyers  would  use 
it  more. 

-most  of  the  delay  reasons  the  buyers  use  do  not  fall  under 
any  of  the  established  codes . 

99.  There  should  be  a  better  system  of  communication  and 
understanding  of  AMIS  from  what  it  does  and  why  it  does  this 
and  what  the  output  is  useful  for. 


Procurement  Analyst  Comments 


100.  As  a  contracting  officer,  I  was  never  adequately 
trained  to  access  the  data  base  nor  made  aware  of  the 
available  capability.  In  some  directorates,  access  was 
restricted  to  clerical  personnel  which  is  totally 
unacceptable.  Within  my  current  directorate,  buyers  are 
inadequately  trained  on  the  completion  of  AMIS  forms. 

101.  AMIS  should  be  designed  so  that  MSD  information  is 
readily  captured  by  AFSC  for  their  "instant  reports." 

102.  Repetitious  questions  on  PMS  BUY  PLANS  and  DD350  and 
67X/67X  forms.  Again,  same  information  is  needed  all  the 
time  for  a  new  modification.  Need  to  develop  a  system  to 
stop  all  this  duplication  of  workl  "Keep  it  Simple." 

103.  This  survey  is  not  accurate  --  it  does  not  take  into 
effect  individual  policies  at  various  activities  nor  does  it 
look  at  equipment/ funds  availability  nor  workload  to 
indicate  which  methods  (decentralized/centralized)  are  more 
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effective.  It  is  evident  that  you  are  striving  for  a 
particular  response  —  surprise! 

104.  All  on-line  access  to  AMIS/PMS  databases  should  be  via 
local  hard-wired  terminals,  with  databases  hosted  on  local 
computer  systems,  thus  eliminating,  problems  with 
communications  (poor  quality  phone  lines,  slow  baud  rates, 
etc) .  Corporate  databases  should  be  updated  overnight  in 
batch  mode  via  long-haul  communications  from  local  systems. 
Existing  software  must  be  replaced  or  made  faster  and  more 
user-friendly.  An  ideal  workstation  would  contain  all 
software  necessary  for  contracting  functions  (i.e. 
spreadsheets,  word  processing,  etc)  along  with  access  to 
AMIS/PMS  databases,  and  printing  capabilities. 

105.  Without  management  experience  and  commitment, 
computerization  of  contracting  will  never  work.  Managers 
need  to  be  committed. 


127 


Bibliography 


1.  Cook,  Lt  Col  Curtis  R.  "The  Aquisition  Management 
Information  System — Friend  or  Foe?"  HQ  Air  Force 
Systems  Command,  Andrews  AFB  MD,  March  1984 
(AD-P002751) . 

2.  Davis,  Gordon  B.  and  Margrethe  H.  Olsen. 

Management  Information  Systems  -  Conceptual 
Foundations.  Structure  and  Development  (Second 
Edition).  New  York:  McGraw-Hill  Book  Company, 

1985  . 

3.  Department  of  the  Air  Force,  Air  Force  Systems 
Command.  Contracting  Automation:  AFSC  Acguisition 
Management  Information  System  (AMIS) .  AFSCR  70-13. 
Andrews  AFB:  HQ  AFSC,  28  August  1987. 

4.  Di liman,  Don  A.  Mail  and  Telephone  Surveys  -  The 
Total  Design  Method.  New  York:  John  Wiley  &  Sons, 
Inc.,  1978. 

5.  Drake,  Daniel  J.  Greater  Buyer  Effectivness  Through 
Aut omat ion .  Logistics  Management  Institute, 

Bethesda  Md,  January  1989  (AD-A206701) . 

6.  Eldin,  Hamed  Kamal .  AMIS  -  Acguisition  Management 
Information  System  Problems  &  Promises.  Report  for 
Air  Force  Business  Research  Management  Center, 
Wright-Patterson  AFB  OH,  29  July  1982  (AD-A146602) . 

7.  Heisey,  Dennis  and  Gerard  Nimis.  STATISTIX:  An 
Interactive  Statistical  Analysis  Program  for 
Microcomputers .  St.  Paul  MN :  NH  Analytical 
Software,  1986. 

8.  Isaac,  Stephen  and  William  B.  Michael.  Handbook  in 
Research  and  Design.  San  Diego:  EDITS  Publishers, 
May  1978. 

9.  United  States  General  Accounting  Office.  Millions 
Wasted'  to  Develop  Major  Energy  Information  System. 
Report  AFMD-81-40.  Washington:  Government  Printing 
Office ,  1981  . 


128 


10.  Whittle,  Lt  Col  Alan  N.,  Director  of  Contracting 
Automation,  Air  Force  Systems  Command.  "Initial 
Strawman  for  Interated  Distributed  System  Design." 
Letter  to  all  Directors  of  Contracting,  Air  Force 
Systems  Command.  16  June  1989. 


♦ 


1 


9 


129 


Captain  Garry  R.Y,  Shafovaloff 


J 


He  served  as  an 

enlisted  member  of  the  United  States  Air  Force  from  February 
1978  until  January  1984.  He  was  selected  as  one  of  the 
Twelve  Outstanding  Airmen  of  the  Air  Force  for  1980.  In 
December  1983  he  graduated  from  Auburn  University  at 
Montgomery  with  a  Bachelor  of  Science  in  Business 
Administration.  Subsequent ly ,  he  received  his  commission  in 
May  1984  from  the  Air  Force  Officer  Training  School.  He 
graduated  from  Officer  Training  School  as  a  Distinguished 
and  Honor  Graduate.  He  served  for  four  years  at  the 
Aeronautical  Systems  Division  at  Wright-Pat t ers on  AFB,  Ohio. 
From  July  1984  through  May  1986  he  was  a  contracts  manager 
for  the  Directorate  of  Research  and  Development  Contracting. 
Beginning  in  June  1986  he  served  as  a  contracts  manager  and 
then  contracting  officer  for  classified  systems  until  he 
entered  the  School  of  Systems  and  Logistics ,  Air  Force 
Institute  of  Technology,  in  June  1988. 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE  omZ^fv-oiaa 

la.  REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

lb.  RESTRICTIVE  MARKINGS 

2a.  SECURITY  CLASSIFICATION  AUTHORITY 

3.  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Approved  for  public  release; 
distribution  unlimited 

2b.  OECLASSIFICATION  /  DOWNGRADING  SCHEDULE 

4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

AFIT/GCM/LS Y/89  S - 1 2 

S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 

6a.  NAME  OF  PERFORMING  ORGANIZATION 

School  of  Systems 
and  Logistics 

6b.  OFFICE  SYMBOL 
(If  applicable) 

AFIT/LSY 

7a.  NAME  OF  MONITORING  ORGANIZATION 

6c  AOORESS  {City,  State,  and  ZIP  Code) 

Air  Force  Institute  of  Technology 
Wright-Patterson  AFB  OH  45433-6503 

7b.  ADDRESS  (City,  State,  and  ZIP  Code) 

8a.  NAME  OF  FUNDING /SPONSORING 
ORGANIZATION 

see  Block  19 

8b.  OFFICE  SYMBOL 
(If  applicable) 
AFSC/PKQ 

9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 

8c  ADDRESS  (City,  State,  and  ZIP  Code) 

Wright-Patterson  AFB  OH  45433-6503 

10.  SOURCE  OF  FUNDING  NUMBERS 

PROGRAM 
ELEMENT  NO. 

PROJECT 

NO. 

TASK 

NO 

WORK  UNIT 
ACCESSION  NO. 

11.  TITLE  (Include  Security  Classification) 

A  NEEDS  ASSESSMENT  OF  AIR  FORCE  SYSTEMS  COMMAND  BUYING  ACTIVITY  AQUISITION 
MANAGEMENT  INFORMATION  SYSTEM  (AMIS)  USERS 

12.  PERSONAL  AUTHOR(S) 

Garry  R.Y.  Shafovaloff,  B.S.,  Captain,  USAF 

1  13a.  TYPE  OF  REPORT  13b.  TIME  COVERED 

!  MS  Thesis  from  to 

4.  DATE  OF  REPORT  (Year,  Month,  Day)  15.  PAGE  COUNT 

1989  September  142 

16.  SUPPLEMENTARY  NOTATION 


|  17.  COSATI  CODES 

18.  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number)  j 

FIELD 

GROUP 

SUB-GROUP 

Acquisition,  Government  Procurement,  Air  Force  j 
Procurement,  Management  Information  Systems,  j 
Data  Bases  j 

15 

05 

05 

01 

1  19.  ABSTRACT  {Continue  on  reverse  if  necessary  and  identify  by  block  number)  E 

Block  8a.  Air  Force  Systems  Command  Directorate  of  Contracting  Automation,  j 

Thesis  Advisor:  Curtis  R.  Cook,  Lt  Col,  USAF  H 

Assistant  Professor  of  Contracting  Management  | 

Department  of  System  Acquisition  Management  j 


Approved  for  nubile  release:  IAW  APR  190-1. 
U  •  .  * 

LARRY  tt.  EMMELHAINZ,  Lt  Cdl,  USAF  11  Oct  89 
Director  of  Research  and  Consultation 
Air  Force  Institute  of  Technology  (AU) 
Wright-Patterson  AFB  OH  45433-6583 


20.  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 

69  UNCLASSIFIED/UNLIMITED  □  SAME  AS  RPT.  Q  OTIC  USERS 

21.  ABSTRACT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Curtis  R.  Cook,  Lt  Col,  Asst  Professor 

22b.  TELEPHONE  (Include  Area  Code) 

(513) 255-3355 

22c.  OFFICE  SYM30L 

LSY 

DO  Form  1473,  JUN  86  Previous  editions  are  obsolete.  SECURITY  CLASSIFICATION  OF  THIS  page 


UNCLASSIFIED 


UNCLASSIFIED 


The  Air  Force  Systems  Command  (AFSC)  Acquisition 
Management  Information  System  (AMIS)  is  continually  being 
improved  to  better  meet  the  needs  of  its  users .  The  primary 
object  of  this  research  was  to  support  the  AFSC  AMIS  program 
office  and  the  AFSC  Contract  Automation  Working  Group  in 
determining  the  buying  activity  users'  level  of  satisfaction 
and  unmet  needs.  The  survey  results  will  provide  AFSC 
additional  information  on  what  improvements  to  AMIS  will 
provide  the  most  utility  to  the  users. 

The  study  found  that  36%  of  AFSC  buying  activity 

users  are  satisfied  with  AMIS.  Forty-four  percent  (44%) 
indicated  dissatisfaction.  Users  believe  that  more  training 
is  needed,  that  the  system  should  be  designed  for  the 
infrequent  user,  and  that  input  redundancies  should  be 
reduced.  Contracting  officers  and  buyers  believe  that  AMIS 
needs  to  provide  more  utility  for  their  functions. 

Management  needs  analysis  tools  . 

Many  of  the  users'  unmet  needs  will  be  provided  by  the 
Integrated  Distributed  System  (IDS) ,  a  major  AFSC  AMIS 
development  project.  All  improvements  to  AMIS  must  "fit" 
the  information  and  automation  needs  of  the  user  and  must 
contribute  to  an  improved  procurement  process. 


UNCLASSIFIED 


